| View previous topic :: View next topic |
| Author |
Message |
madpenguin8
n00b
Joined: 19 Jun 2002
Posts: 38
Location: Detroit MI
|
 Posted: Sun Feb 02, 2003 9:15 pm Post subject: Gentoo forum search sucks |
 |
|
Anyone else think that the gentoo forum search sucks. It comes up with just about nothing relevant to my search. It seems like really popular posts that have absolutely nothing to do with what I am searching for always turn up close to the top. Writing up a dupe sucks, but what can a guy do if search turns up garbage.
_________________
Have you ever stopped to think...............and never start again? |
|
| Back to top |
|
 |
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Sun Feb 02, 2003 9:20 pm Post subject: |
|
|
Some tips: make sure your search terms are all 3+ letters long. Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want. Use many anded search terms, and widen the search later. Try to choose obscure words if you can: unusual words that appear in an exact error message are usually very good. Things like "bug", "portage", "broken" are bad.
Do you have any recommendations for improving the search engine while still being able to upgrade phpBB in a timely fashion?
EDIT: To consolidate some search threads, I'm including the following quote from another thread. -- pjp | rac wrote: | Here's how the phpBB search function works. Each post is split into words. First, some characters are replaced. There are three classes of characters here, those that get replaced by spaces, those that get elided, and those that get left alone. Next, whitespace is used to delineate words. All words of less than 3 or more than 20 characters are dropped. Then an entry is made in the dictionary table for every word that is not in the dictionary, so that it can be referenced by number. An entry is made in a colossal table for each and every word in each and every post. That's what gets searched against.
To get back to Reformist's two examples, gnome2 is a word. 'gnome 2' is two words, one of which is impossible to match because it is one character long. "1.1.0" is three words, each of which are impossible to match, because they are one character long. One modification that it might be feasible to make would be to change the status of '.'. If it were left alone, version numbers would become searchable. However, words at the end of sentences, followed by periods, would become unsearchable, because a separate entry would be made including the period. If it were elided, the end-of-sentence problem would go away, but then you would have to search for "abiword and 110", and "2.1" would become "21" and fall under the three-character limit. |
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
PARENA
Guru
Joined: 06 Jan 2003
Posts: 349
|
| Posted: Sun Feb 02, 2003 9:55 pm Post subject: |
|
|
Here's an example of a (what I think) good search. Let's say you want to know about using the slit in Openbox:
openbox AND slit
(almost) guaranteed to come up with your answer. Unless it's not on the forum of course.  |
|
| Back to top |
|
|
idl
Retired Dev
Joined: 24 Dec 2002
Posts: 1728
Location: Nottingham, UK
|
| Posted: Sun Feb 02, 2003 10:42 pm Post subject: |
|
|
| would be cool if we could do the old "kde3.1 emerge failure" note the "" that makes sure it matches the whole string. like on google |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Sun Feb 02, 2003 11:50 pm Post subject: |
|
|
| port001 wrote: | | would be cool if we could do the old "kde3.1 emerge failure" note the "" that makes sure it matches the whole string. like on google |
That, while a nice feature, is completely impossible with the current way the search databases work, because the search match tables have fields for word number and post id only. There is no sense of what words occur next to one another.
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
Matje
l33t
Joined: 29 Oct 2002
Posts: 619
Location: Hasselt, Belgium
|
| Posted: Mon Feb 03, 2003 12:35 am Post subject: |
|
|
| rac wrote: | | port001 wrote: | | would be cool if we could do the old "kde3.1 emerge failure" note the "" that makes sure it matches the whole string. like on google |
That, while a nice feature, is completely impossible with the current way the search databases work, because the search match tables have fields for word number and post id only. There is no sense of what words occur next to one another. |
It _is_ possible, by making php look into the actual post texts. Would be a query like:
SELECT post_id FROM phpbb_posts_text WHERE post_text LIKE '%$searchstring%'
But this'll butcher the mysql I think with these number of posts on the forum
_________________
Life is like a box of chocolates... Before you know it, it's empty... |
|
| Back to top |
|
|
xlyz
Veteran
Joined: 27 Oct 2002
Posts: 1470
Location: Italy
|
| Posted: Mon Feb 03, 2003 2:08 am Post subject: |
|
|
| rac wrote: | | Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want. |
please make "AND" default. "OR" is seldom used |
|
| Back to top |
|
|
gsfgf
Veteran
Joined: 08 May 2002
Posts: 1266
|
| Posted: Mon Feb 03, 2003 3:01 am Post subject: |
|
|
| xlyz wrote: | | rac wrote: | | Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want. |
please make "AND" default. "OR" is seldom used |
at least for the quick search. |
|
| Back to top |
|
|
pilla
Bodhisattva
Joined: 07 Aug 2002
Posts: 7731
Location: Underworld
|
| Posted: Mon Feb 03, 2003 3:07 am Post subject: |
|
|
Moving to Gentoo Forum Feedback.
_________________
"I'm just very selective about the reality I choose to accept." -- Calvin |
|
| Back to top |
|
|
Matje
l33t
Joined: 29 Oct 2002
Posts: 619
Location: Hasselt, Belgium
|
| Posted: Mon Feb 03, 2003 10:45 am Post subject: |
|
|
| gsfgf wrote: | | xlyz wrote: | | rac wrote: | | Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want. |
please make "AND" default. "OR" is seldom used |
at least for the quick search. |
Indeed, I'd like that too, would be a great improvement.
_________________
Life is like a box of chocolates... Before you know it, it's empty... |
|
| Back to top |
|
|
David_Escott
l33t
Joined: 12 Jan 2003
Posts: 952
Location: Boston, MA
|
| Posted: Mon Feb 03, 2003 1:37 pm Post subject: |
|
|
I have the compelling urge to say the following:
THIS IS A DUPLICATE THREAD.
PLEASE SEARCH
https://forums.gentoo.org/viewtopic.php?t=30782
was the closest I could get, searching was a pain because well it sucks (sorry that was a duplicate thought)
But I do remember seeing one thread where rac may have explained some of the difficulties in trying to make search work a little better, if only I could find it again :(
Ahhh found it
https://forums.gentoo.org/viewtopic.php?p=167526#167526 down the page rac explains some of the difficulties |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Mon Feb 03, 2003 10:13 pm Post subject: |
|
|
Thanks for those links, David_Escott. As a journey of a thousand miles begins with a single step, I have changed the default setting of the checkbox in the search screen to "Search for all terms". Hopefully this will improve things in some small measure. Note that this change only affects the purple "gentoo" theme - people still using subSilver will not be affected.
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
xlyz
Veteran
Joined: 27 Oct 2002
Posts: 1470
Location: Italy
|
| Posted: Mon Feb 03, 2003 10:51 pm Post subject: |
|
|
quick search is still "search for any words" as default
are you going to change it as well?
TIA |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Mon Feb 03, 2003 11:30 pm Post subject: |
|
|
Apparently phpBB was caching the template file, so my changes to quicksearch weren't taking hold. It should be fixed now.
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
gsfgf
Veteran
Joined: 08 May 2002
Posts: 1266
|
| Posted: Mon Feb 03, 2003 11:40 pm Post subject: |
|
|
| Rac i noticed in yopur other post that version #s aren't searchable. If you convert 2.2.2 to 222 and make search strip periods as well so if you search for kde 3.1 search will treat it as kde 31, that would solve that issue. That may be harder than it looks, though. |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Mon Feb 03, 2003 11:48 pm Post subject: |
|
|
| gsfgf wrote: | | If you convert 2.2.2 to 222 and make search strip periods as well so if you search for kde 3.1 search will treat it as kde 31, that would solve that issue. That may be harder than it looks, though. |
It is harder than it looks, for a couple of reasons. Either a period causes a word break or it doesn't. Now what would be best is if it caused a word break only if it wasn't a version number, but that could be a challenging regex. Maybe we could steal it from Portage. I think if we're going to go this far, we might as well get it right and have "version numbers" go into the index.
I just had an idea on how this might be implemented. Details later.
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
phong
Bodhisattva
Joined: 16 Jul 2002
Posts: 778
Location: Michigan - 15 & Ryan
|
| Posted: Wed Feb 05, 2003 4:35 am Post subject: |
|
|
Not that tough of a regex... For matching (in a generic way) words of 3+ characters and version numbers, I might use the following for matching words to put into the index (offhand, could be more robust):
(\b\w{3,}\b|\b\d(\.\d){1,2}\b)
What's the current regexp?
_________________
"An empty head is not really empty; it is stuffed with rubbish. Hence the difficulty of forcing anything into an empty head."
-- Eric Hoffer |
|
| Back to top |
|
|
Lion
Apprentice
Joined: 23 Jun 2002
Posts: 207
|
| Posted: Sat Feb 08, 2003 10:07 am Post subject: Search still broken |
|
|
I think there is still something basically wrong with search.
I always try to include as many relevant search terms in my query, but still I do not get posts that I know to be available.
Simple example: Search for the word 'world'.
'No topics or posts met your search criteria'.
I know this is not true, because the world file is mentioned in many posts.
Search for 'world AND file'.
Thousands of posts, many of which do NO contain the word 'world'.
So, my question is: what am I doing wrong? |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Sat Feb 08, 2003 7:48 pm Post subject: Re: Search still broken |
|
|
| Lion wrote: | | So, my question is: what am I doing wrong? |
Being unlucky. To help keep the size of the search tables down, there is a "stopword list" in phpBB's search function. Words on the stopword list are not indexed because they are too common. Unfortunately for your example, world is on the stopword list, so nothing shows up, and then when you search for "world and file", you are really only searching for 'file'.
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20554
|
| Posted: Sat Feb 08, 2003 8:02 pm Post subject: |
|
|
Maybe emerge, portage, gentoo, gnome and alot of others should be added to the list 
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
xlyz
Veteran
Joined: 27 Oct 2002
Posts: 1470
Location: Italy
|
| Posted: Sat Feb 08, 2003 8:59 pm Post subject: Re: Search still broken |
|
|
| rac wrote: | | Lion wrote: | | So, my question is: what am I doing wrong? |
Being unlucky. To help keep the size of the search tables down, there is a "stopword list" in phpBB's search function. |
what are the words included in the list? |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Thu Feb 13, 2003 2:45 am Post subject: Re: Search still broken |
|
|
I assume that gentoo bugzilla has a similar stopword list? It certainly would explain some of my difficulties in using search there.
Do you have an inkling (and could you share the location) of the search stopword list there?
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
rac
Bodhisattva
Joined: 30 May 2002
Posts: 6553
Location: Japanifornia
|
| Posted: Thu Feb 13, 2003 6:04 pm Post subject: Re: Search still broken |
|
|
| dufeu wrote: | | I assume that gentoo bugzilla has a similar stopword list? |
Bugzilla's completely different software. I don't know off the top of my head whether there's a stopword list. If I get some time I may look into it further.
_________________
For every higher wall, there is a taller ladder |
|
| Back to top |
|
|
edoloughlin
n00b
Joined: 02 Oct 2002
Posts: 12
Location: Ireland
|
| Posted: Wed Feb 19, 2003 10:16 am Post subject: Re: Search still broken |
|
|
| rac wrote: | | Lion wrote: | | So, my question is: what am I doing wrong? |
Being unlucky. To help keep the size of the search tables down, there is a "stopword list" in phpBB's search function. Words on the stopword list are not indexed because they are too common. Unfortunately for your example, world is on the stopword list, so nothing shows up, and then when you search for "world and file", you are really only searching for 'file'. |
Perhaps a lesson could be learned from Google. Stopwords are identified if they are included in a search, viz:
| Quote: | | "the" is a very common word and was not included in your search |
|
|
| Back to top |
|
|
|
Gentoo Forums Feedback
