Interview with DuoSearch co-founder Justin Drake, the best web based OpenBazaar search engine

We interviewed Justin Drake, co-founder of DuoSearch, along with Daniel Murrell, both Cambridge alumni and armed with degrees in mathematics, computer science and a machine learning PhD have founded one of the first companies to create a consumer friendly index and search engine for OpenBazaar stores and listings.

1. How are you dealing with illegal listings found? We believe this is always one of the biggest issues a company indexing distributed content faces. In the case of store listings, how are you protecting your users from not just finding illegal products, but from fraudulent sellers?

That’s a hard first question. From a purely legal standpoint we’re a UK limited company and as such have some legal liability. From a more holistic standpoint the guiding principle for Duo Search is user experience; compliance is a somewhat secondary necessity. We want our users to feel comfortable using our search. If a listing is judged to be detrimental to the search experience for a typical user, it will not be included by default in our search results. We’ve done this for listings tagged NSFW (as of writing 78 listings out of 4734 online listings).

Going back to legal considerations, we have looked at other search engines for precedence. Things like the DMCA, the right to be forgotten, and the Pirate Bay legal issues are learning points. In the short term, we will handle complaints and attacks as they come and learn from those. In any case, Duo Search will fight for inclusiveness of listings that do not harm user experience, and we will be transparent about listings that do end up censored.

Dealing with fraudulent sellers is definitely something we take seriously as they are a big liability to user experience. Duo Search takes a zero-tolerance policy towards fraudsters. We will build infrastructure for users to flag fraudsters. One of the Duo Search founders has applied machine learning during his PhD and will train a learning algorithm to detect potentially fraudulent vendors.

2. Are you seeing growth in the number of new stores and listings found by your crawlers?

The number of listings has been growing at a fast pace. Today we see about 4,500 online listings when three weeks ago it was about 3,000. The number of stores is not growing as fast. Three weeks ago there were about 250 online vendors with at least one listing, and now about 300. Store professionalism is increasing fast though, as test stores become real stores.

3. How are you keeping your index up to date? how often do you crawl stores you’ve previously crawled?

We have a relatively heavy-weight infrastructure for crawling and scraping nodes based on a farm of about 10 well-connected nodes around the world. Crawling is done using a combination of techniques, some passive and some aggressive. We generally know about node connections and disconnections (for both vendors and non-vendors nodes) within a handful of seconds.

Known vendors are pinged and scraped continuously for changes. We strive to update our search engine within seconds of a store update, and we’re working with the OpenBazaar dev team to make protocol changes to make this as non-burdensome as possible for stores. Our pingers also keep track of store uptime and responsiveness. Stores with high uptime and low ping latency will soon rank higher in our searches.

4. How long does it take nowadays to crawl the entire OpenBazaar network? (this will be a cool question to ask you a few months from now. As node count grows, it gets harder and harder to take snapshots of the distributed network).

Crawling the entire OpenBazaar network right now takes little time. It’s a continuous process, but if we started from scratch is would take just a few minutes. Handling hundreds of thousands of listings will be a fun challenge. We’re gearing up for scale.

5. Will you be offering premium listing services or sponsored search results any time soon?

Duo Search will not be offering premium listings or sponsored search results soon. We’re first and foremost focused on delivering the best OpenBazaar experience for buyers. We may implement premium listings if it promised to be an effective way to promote high quality listings to buyers.

6. Will you be implementing price comparisons?  or to let users review their experience with stores (given they can prove a purchase has been made)? We believe the best search engine for Openbazaar could position itself as THE reputation broker for sellers network wide.

Price comparisons is an interesting idea which we haven’t prioritised yet. So far we’re seeing mostly boutique stores with handmade items or ad-hoc services. We’re looking forward to bulk sellers offering commodities at a large discount compared to Amazon prices.

7. Technology: What are you using to index all the content? Elastic Search/Apache Lucene/Solr?

None of the above. It’s a proprietary solution designed for speed.

8. Are you hosting your services with any major cloud provider?

Yes. We use DigitalOcean and MaxCDN.

9. Are you looking for investors? Are you hiring?

We’re not actively looking for further investors. (We’ve raised a seed round and have sufficient cash.) The team is currently just two founders and we’re looking to make a first quality hire. The OpenBazaar talent pool is small but we feel confident we will attract some of the best minds in the space to join our exciting journey.

10. Where in the UK are you based and what is your background before this venture?

The team is based in Cambridge, and both founders (Justin Drake and Daniel Murrell) are alumnus of the university. I did my undergrad and masters’ in mathematics and was a high-tech consultant for 3 years. Daniel did a PhD in machine learning. The team is quite international. I previously lived in France and China, and Daniel in South Africa.

ADVERTENCIA: Este es un artículo de carácter informativo. DiarioBitcoin es un medio de comunicación, no promociona, respalda ni recomienda ninguna inversión en particular. Vale señalar que las inversiones en criptoactivos no están reguladas en algunos países. Pueden no ser apropiadas para inversores minoristas, pues se podría perder el monto total invertido. Consulte las leyes de su país antes de invertir.