Bug Report

Ineffective search [#1623]

Back to forum

New reply

 
reply
Unkx80: Ineffective search (2008-11-17 14:10) [#5280]

Copied from topic:1622:

The SL search is very inefficient. google's site:[ext] http://senseis.xmp.net/ tends to be better... At the very least, when I use a particular phrase or collection of keywords, then always should SL list a page with 100% matching at place 1. It does not. E.g., searching for "Ko Types" lists KoTypes only as entry 31!

My hunch is that the current SL search engine only indexes the page contents but not the titles. From what I know, it is powered by Xapian, and a quick look at its documentation seems to suggest that it does not support indexing multiple fields (e.g., title and text).

What should have been done is to index multiple fields simultaneously, and to heavily weigh the matches from the title field. I think Lucene is pretty good at this aspect, but I think you might have to perform word stemming yourself when using Lucene. One thing though, Lucene is implemented in Java and I do not know how easy is it to perform Java-PHP integration.

reply
ArnoHollosi: improving search results (2008-12-14 23:18) [#5414]

I spent most of the day improving search results in different ways. I even coded a Google style Pagerank algorithm (not active).

I finally settled for a tweak in Xapian's parameters and weights for the full text sort order (Xapian is the underlying search enginge used by SL).

Please give it a try and let me know what you think about the search results.

@unkx80: Xapian is comparable to Lucene. The full text index adds the title, titles of alias pages, the keywords, even the links etc. In addition, one can access all these features through (undocumented(*)) prefixes. Actually, one could do most of AdvancedFindPage through the Xapian interface (e.g. try "+keyword:joseki +difficulty:Expert +title:pincer")

(*) undocumented on purpose, subject to change without notice.

X
Unkx80: Forum search? (2009-01-04 19:05) [#5515]

Hi Arno,

The new search seems much improved over the previous search. Thanks.

However, is it possible to search forum posts?

 
Back to forum

New reply


[Welcome to Sensei's Library!]
RecentChanges
StartingPoints
About
RandomPage
Search position
Page history
Latest page diff
Partner sites:
Go Teaching Ladder
Goproblems.com
Login / Prefs
Tools
Sensei's Library