phpWebLog

By Twyst on Thursday September 21 2000 @ 01:38PM EDT [ Development ]
Well, I was thinking about all the problems I've been having with setting up a link directory... and then it came to me.. Why not use the ODP data? There's a huge number of links in the Open Directory Project, and it's free for use!

Now, there is a php class that is freely available (GPL), called phpODP - It's very simple, and being a class, can be used easily enough. Why not expand and include this in PHPweblog?

The class does have a few things that still need to be done with it.. but it should work for the most part. It needs a search capability, and a way of choosing where it starts in the directory (ie. /Arts/Animation/Anime)

Anyways, that's my latest project. If you're interested, I'll let you know when I have all the functions working!

< Feature enhancement possible? | RDF >

mocha writes on Friday September 22 2000 @ 11:35AM EDT: [ reply | parent ]

i believe phpodp parses on a page per page basis from the ODP servers. i think what you are looking to do is parse the DATA FILES into a db?

Twyst writes on Friday September 22 2000 @ 03:09PM EDT: [ reply | parent ]

Yes, it does grab the live data. The data files are all well and good, but it's a 100+ meg download, and they aren't as current as the live data.

mocha writes on Saturday September 23 2000 @ 04:42PM EDT: [ reply | parent ]

the problem with parsing live data is that it's:

1. slower
2. depends on their server, if their server is bogged down or is offline for some reason, so will your access.

as you pointed out, you do get the latest version of their data though. for me, i've choosen to go with grabbing the data files, and parsing that into mysql. but then again, i only needed a branch of their data files.

Twyst writes on Sunday September 24 2000 @ 02:40AM EDT: [ reply | parent ]

Well, I only need a branch of them as well. Got any pointers? I'll do the full download if there's a reasonable way to implement it..

Anonymous writes on Sunday December 23 2001 @ 03:58PM EST: [ reply | parent ]

Where can I get the odp data files via ftp?

Gregory writes on Tuesday December 25 2001 @ 05:52PM EST: [ reply | parent ]

ODP data files are really big and their are a lot of them. If I where you I would just parse the search results. If you index sites you have to maintain the databases and trust me it's real pain.

Merry Christmas Everyone

Gregory :)

Gregory writes on Tuesday December 25 2001 @ 05:57PM EST: [ reply | parent ]

The ### is just to fool the parser delete

(.*?)(.*?)/";

$eng[1]['url'] = "http://www.alltheweb.com/cgi-bin/search?offset=0&type=all&query=$q";
$eng[1]['name'] = "Fast";
$eng[1]['reg'] = "/.*?status=\'(.*?)\';.*?true\">(.*?).*?=t>(.*?)/";

$eng[2]['url'] = "http://search.dmoz.org/cgi-bin/search?search=$q";
$eng[2]['name'] = "Open";
$eng[2]['reg'] = "/(.*?)\ -(.*?)/";

$eng[3]['url'] = "http://altavista.com/sites/search/web?q=$q&enc=&kl=XX&search=Search";
$eng[3]['name'] = "Altavista";
$eng[3]['reg'] = "/(.*?).*?(.*?)/i";
$eng[3]['strip'] = "\n";

/*
// May not be working
// $eng[4]['url'] = "http://search.excite.com/search.gw?c=web&search=$q&onload=";
// $eng[4]['name'] = "Excite";
// $eng[4]['reg'] = "/pos=.*?;(http.*?)onMouse.*?(.*?).*?size8>(.*?)/i";
// $eng[4]['strip'] = "\n";

// Reduced number of engines to increase page load time.
// $eng[4]['url'] = "http://search.msn.com/results.asp?ba=(0.0)0&co=(0.10)4.1..2.1.4.1&FORM=MSNH&RS=CHECKED&q=$q&v=1";
// $eng[4]['name'] = "MSN";
// $eng[4]['reg'] = "/(.*?)(.*?)(.*?).*?(.*?)/i";
// $eng[6]['strip'] = "\n";
*/

$urls = array();
$out = array();
$y=0;

foreach($eng as $c)
{
$text1 = join("",file($c['url']));

if($c[strip])
$text1 = ereg_replace($c[strip],"",$text1);

preg_match_all($c['reg'],$text1,$matches);
$num = count($matches[0]);

for($x=0;$x";
print "";
#print "";
#print_r($out);
#print "";

while(list($url,$rec) = each($out))
{
extract($rec);
print "

$title
$description
$url
$engine
";
}

print "";
?###>

Search:

Gregory writes on Tuesday December 25 2001 @ 06:04PM EST: [ reply | parent ]

Sorry about my posts code did not parse properly. due to stripslashes() in the parser.inc If you need a search engine parser let me know @ http://www.developers-resources.com/phpBB/ and I will post the code on my site for you.

Gregory

each site writes on Sunday October 17 2004 @ 06:45PM EDT: [ reply | parent ]

You are invited to visit some information about go there | go there | http://www.viagra-use.com/ | come here | come here | http://www.viagra-100-deals.com/ | each site | each site | http://www.phrensy.org/ | come here | come here | http://www.pages4people.com/ | every link | every link | http://www.worldwide-sites.net/ | following link | following link | http://www.diet-pills-4me.net/ | that site | that site | http://www.casino-4u.net/ | this link | this link | http://www.poker2004.net/ | link | link | http://www.poker-4u.net/ | here | here | http://www.onlinepokersite.net/ | home of | home of | http://www.worldwide-em.com/ | click here | click here | http://www.phentermine-375-deals.com/ | everywhere | everywhere | http://www.taliesinfellows.org/ | everywhere | everywhere | http://www.bestdeals-winner.net/ | goto | goto | http://www.online-casino-winner.net/ | ... Thanks!!!

biaxin xl writes on Saturday October 23 2004 @ 06:07AM EDT: [ reply | parent ]

You may find it interesting to check some information in the field of custom poker chips | custom poker chips | http://www.custom-poker-chips-2000.info/ | casino craps | casino craps | http://www.list-1999.info/ | poker tournament | poker tournament | http://www.poker-tournament-guide.info/ | poker chip cases | poker chip cases | http://www.poker-chip-cases-4u.info/ | buy prescription drug online | buy prescription drug online | http://www.top-deal-pills.info/ | glipizide | glipizide | http://www.glipizide-pharmacy.info/ | vaniqa | vaniqa | http://www.top-shop-site-medication.info/ | blood pressure medicine | blood pressure medicine | http://www.top-purchase-pharmacy.info/ | strip poker | strip poker | http://www.win-anytime-4u.info/ | cheap bontril | cheap bontril | http://www.cheap-bontril-online-pharmacy.info/ | blackjack card counting | blackjack card counting | http://www.online-site-2001.info/ | buy lipitor | buy lipitor | http://www.best-online-shop-2002.info/ | ...

biaxin xl writes on Saturday October 23 2004 @ 06:07AM EDT: [ reply | parent ]

You may find it interesting to check some information in the field of custom poker chips | custom poker chips | http://www.custom-poker-chips-2000.info/ | casino craps | casino craps | http://www.list-1999.info/ | poker tournament | poker tournament | http://www.poker-tournament-guide.info/ | poker chip cases | poker chip cases | http://www.poker-chip-cases-4u.info/ | buy prescription drug online | buy prescription drug online | http://www.top-deal-pills.info/ | glipizide | glipizide | http://www.glipizide-pharmacy.info/ | vaniqa | vaniqa | http://www.top-shop-site-medication.info/ | blood pressure medicine | blood pressure medicine | http://www.top-purchase-pharmacy.info/ | strip poker | strip poker | http://www.win-anytime-4u.info/ | cheap bontril | cheap bontril | http://www.cheap-bontril-online-pharmacy.info/ | blackjack card counting | blackjack card counting | http://www.online-site-2001.info/ | buy lipitor | buy lipitor | http://www.best-online-shop-2002.info/ | ...

play poker writes on Monday October 25 2004 @ 01:14AM EDT: [ reply | parent ]

Please check some information dedicated to prescription drug online | prescription drug online | http://www.prescription-drug-online-online-pharmacy.info/ | allergy treatment | allergy treatment | http://www.allergy-treatment-online-pharmacy.info/ | canada online drug store | canada online drug store | http://www.canada-online-drug-store-online-pharmacy.info/ | keno game | keno game | http://www.play-reviews.info/ | buy levitra online | buy levitra online | http://www.buy-levitra-online-pharmacy.info/ | valtrex | valtrex | http://www.valtrex-pharmacy.info/ | tiazac | tiazac | http://www.tiazac-pharmacy.info/ | best price viagra | best price viagra | http://www.best-price-viagra-online-pharmacy.info/ | blackjack rules | blackjack rules | http://www.blackjack-rules-2000.info/ | tramadol | tramadol | http://www.tramadol-pharmacy.info/ | valium prescription | valium prescription | http://www.top-deal-2000.info/ | ...

Post a Comment