|
|
|
|
Burien, Wash. - March 15th, 2005 -- Nathan Enns has interviewed Mark Maunder, the co-founder of the job search engine WorkZoo for publication on The Search Industry Blog owned by FyberSearch. WorkZoo was founded by Mark Maunder and Kerry Boyte who are husband and wife. I was very impressed by the job search technology they have put together and am excited that Mark had enough time to do this interview with me (Nathan Enns) for The Search Industry Blog. Question #1 Nathan Enns: What motivated you and Kerry decide to start your own job search engine and what is your background in the search industry? Mark Maunder: Kerry and I both have backgrounds in software consulting. While we were both consulting in London we would keep an eye on most of the job boards and it became time consuming. So in our spare time we set up a meta-search that searched the top 10 UK job boards and combined the results. The site back then was very basic, but we built up a small user base. In 2003 we moved to California and started looking for business ideas. The USA has more than 100 times the number of job boards in the UK and WorkZoo seemed like an obvious fit. So in 2004 I started full-time work on a US version and changed WorkZoo from a meta-search to a full-blown search engine with it's own crawler and index. Question #2 Nathan Enns: When was WorkZoo officially launched? Mark Maunder: The original UK version of WorkZoo was launched in late 2001. The US version has been around as a meta-search since early 2004 and we officially launched the new search engine with its own index on 24 February this year. Question #3 Nathan Enns: You recently announced that WorkZoo is able to index about 45,000 new job listings a day. Without revealing any secrets, can you tell us about how it is able to process that much data every day? Mark Maunder: Just to clarify, that is the number of jobs we are currently indexing per day. We can handle 5 times that with our existing server capacity. The first stage is our crawler that actually fetches each job from the job boards. The crawler is called ZooBot and is multi-threaded so it fetches multiple jobs simultaneously without overloading any one web site. We respect the robots.txt exclusion standard on all the sites we crawl and provide detailed instructions on our web site for webmasters to limit the amount of data we display for their site or to limit the rate at which we crawl them. As each job is fetched, we do some initial processing on the HTML and we compress the HTML and store that and a plain-text version of the page ready for the indexer. The compressed HTML is what we used to provide cached versions of each job. The next stage is an indexer that crunches through the job data created by the crawler and creates the final index we use for searching. The indexer determines the geographic location (longitude and latitude) of each job and indexes all the words in the job title and body. We continually check job boards for new jobs throughout the day and have made our crawler intelligent enough so that it only fetches a small number of pages if there are no new jobs. In other words we don't crawl an entire job site just to find that there is only one new job. All the above runs on a single server. Every hour we copy the latest index to our search server (or servers if we are clustering) so that our users get the latest jobs with the minimum of lag time. The complete interview that includes all 10 questions can be found at the search industry blog owned by FyberSearch at http://blog.fybersearch.com/interview-with-mark-maunder.php About FyberSearch: FyberSearch is a search engine technology company founded in November 2003 by Nathan Enns. FyberSearch operates several websites that provide Internet users with information processing services and website owners with tools to help them optimize/promote their website. FyberSearch's mission is to "provide revolutionary information retrieval and processing tools to all Internet users". Nathan Enns, Owner and Founder Company: FyberSearch URL: http://www.fybersearch.com Contact: http://www.fybersearch.com/contact.php ### |
| © 2012 FyberSearch - Contact - Privacy Policy - Terms of Service |