Search and forum behavior

[raysun]

I do not intend this to be a rant post...

The forum's search feature has traditionally been inadequate, and rarely returns the posts I am looking for. That's not unusual for forum search engines, rarely do I find them useful.

My standard fallback is to use Google, and navigate to the posts of interest by the URLs returned. The Outback forum Viewpost function supported this quite neatly in the past.

Now with the switch to the new forum server, the Viewpost URLs are not working, and default to the main index page.

Can the Viewpost function be returned to its former functionality for external queries? (I suspect an index problem.)

[provo]

Ouch! I second your concern!

Using Google search and including "Outback Power" in the search line always worked well. To see the new problem, do a search on "outback power view topic". You'll see several hits, and even be able to read a bit of them, but clicking on anything just takes you to the main forum page, as raysun says.

[BruceS]

It'll take all the search engine (& in particular Google) to re-index all the data.
Some search engines look for changes daily, some might be weekly etc.

Cristian?? https://www.reliablesoft.net/force-google-crawl-a-site/

[David LeBow]

The forum's search feature has traditionally been inadequate, and rarely returns the posts I am looking for. That's not unusual for forum search engines, rarely do I find them useful.

Let me see what I can do to address (ok, "address") these issues. First: whatever search functionality the underlying forum engine (here, phpBB) provides us is what we have. Improving upon that is outside the scope of our day jobs. But: see also below.

My standard fallback is to use Google, and navigate to the posts of interest by the URLs returned. The Outback forum Viewpost function supported this quite neatly in the past.

Now with the switch to the new forum server, the Viewpost URLs are not working, and default to the main index page.

Unfortunately, I'm not really clear on what you're referring to with the "OutBack Forum Viewpost function" and "Viewpost URLs". Could you put some additional meat on those diagnostic bones for me?

Can the Viewpost function be returned to its former functionality for external queries? (I suspect an index problem.)

What I can say: we haven't specifically done anything to undo or worsen functionality. We did update to the latest version of the phpBB engine during the shift to a new hosting mechanism, since the patched versions always contain security fixes. It's not impossible that some setting on the new server isn't the same as on the old - that's what happened with the document download. Some files worked, others didn't: turns out that case sensitivity wasn't configured identically on the new server, so filenames didn't match 100% (OutBack <> outback).

It's not impossible that there's some similar setting we need to dig into. We did take the opportunity to rebuild all of the indices while we had the site down.

A lot of words, no real solution. If you could give me any details on what used to work and examples of what aren't, I can see if we can dig in deeper.

[raysun]

Yes, the forum software's search engine is what we are talking about here. I know of no forum software that includes a search engine that returns excellent results. Mediocre seems to be the quality target for forum developers.

Hence the reliance on outside search engines like google. Below is a result from a typical "Forum search" via google:

Sorry for the confusion in nomenclature. The actual command is "viewtopic". When clicking in the search result:

The forum returns the main page. No errors, which leads me to believe this is default behavior when the record cannot be found in the forum. A typical reason for a "no records returned" condition is a missing or corrupt index. The admins might be able to check on that.

[David LeBow]

For my own understanding (and just as much for the web archaeologists who come behind me). I tried googling this: "not feeding the grid outback" in Google. That turned up the following:

Copying the link address gets me this: http://outbackpower.com/forum/viewtopic.php?t=14872

Clicking that link on the google page does indeed take me to the forum index. Bad.

If I copy/paste that link to another browser page... it gets me to the same place. Bad.

Explicitly changing the http to https didn't improve anything, though I didn't really expect that it would. Still bad.

Aha... getting somewhere. If I navigate through the forum index to some post, that uses the following link structure: https://forum.outbackpower.com/viewtopic.php?f=7&t=2771

It looks like google is still holding the olde style link layout from the last time it indexed our site. The same is probably true of other search engines - until we're reindexed.

@Cristian: is there a way to poke search engines in the eye in order to "request" a re-index pass for restructurings of this type?

Then, to complete the pass, I went and searched for "not feeding the grid" (dropping "OutBack") in the forum's search box and came up with - among other things - this:

So: Google 0, OutBack Forum Search 1 at the end of the 3rd quarter. I'm guessing that we need to somehow "request" a re-index or we just need to wait for the forum to come around again on the geetar and be sucked up into the search engine maw.

Make sense?

[raysun]

For my own understanding (and just as much for the web archaeologists who come behind me). I tried googling this: "not feeding the grid outback" in Google. That turned up the following:

ForumSearch1.jpg

Copying the link address gets me this: http://outbackpower.com/forum/viewtopic.php?t=14872

Clicking that link on the google page does indeed take me to the forum index. Bad.

If I copy/paste that link to another browser page... it gets me to the same place. Bad.

Explicitly changing the http to https didn't improve anything, though I didn't really expect that it would. Still bad.

Aha... getting somewhere. If I navigate through the forum index to some post, that uses the following link structure: https://forum.outbackpower.com/viewtopic.php?f=7&t=2771

It looks like google is still holding the olde style link layout from the last time it indexed our site. The same is probably true of other search engines - until we're reindexed.

@Cristian: is there a way to poke search engines in the eye in order to "request" a re-index pass for restructurings of this type?

Then, to complete the pass, I went and searched for "not feeding the grid" (dropping "OutBack") in the forum's search box and came up with - among other things - this:

ForumSearch2.jpg

So: Google 0, OutBack Forum Search 1 at the end of the 3rd quarter. I'm guessing that we need to somehow "request" a re-index or we just need to wait for the forum to come around again on the geetar and be sucked up into the search engine maw.

Make sense?

Yes, makes sense.

Another knuckleball:
When searching the forum using the internal search engine, I can't figure out how to get results other than posts from 2020. The prior years don't seem to be included.

For example: From the topmost Forum Index, searching for FX3048T (with search parameter "ALL" selected for the time frame), Search returns 33 results, the oldest being January 2020.

[David LeBow]

<hits the knuckleball into the dirt, 0 and 1>: Do you know of any posts pre-2020 which should match that? (I know... it's Search which should tell you that, but... just trying to narrow things down for the web archaeologist...).

[raysun]

<hits the knuckleball into the dirt, 1 and 0>: Do you know of any posts pre-2020 which should match that? (I know... it's Search which should tell you that, but... just trying to narrow things down for the web archaeologist...).

Here's one:
viewtopic.php?p=53190#p53190

Also, if a search is done on Author, all posts they have created are returned.

Search done on Keyword and Author restricts results to 2020.

[David LeBow]

Welp... I've scoured all the possible board settings and googled my brains out. I don't see any really clear explanation for this. But: I've deleted and am now recreating the board's search engine index. Not sure how long that will last. Could easily be some number of hours with a board of this size.

Here's hoping for some eventual improvement.

[raysun]

Thanks David!

[David LeBow]

Well... don't thank me yet. I see that the index rebuild is still ongoing. Not sure if it'll get better or not...

[David LeBow]

Yikes... seems like that index is still rebuilding!

[raysun]

Yikes... seems like that index is still rebuilding!

There's a lot of posts!

[fcwlp]

Maybe it is the re-indexing. I just searched for something and only got results from 2003-2005.

[raysun]

Maybe it is the re-indexing. I just searched for something and only got results from 2003-2005.

That's a start! Literally.

[raysun]

As indexing chugs along to its, hopefully, complete and uncorrupted future state, it shows interesting promise.

Search for Lithium.

Sort of Alpha and Omega I'd think.

[David LeBow]

Seems like the @#@$ thing is still indexing..!

Hope the results are getting better.

D.

[raysun]

Seems like the @#@$ thing is still indexing..!

Hope the results are getting better.

D.

Indexing can be a 'watching paint dry' affair.

Searching "Lithium" the other day returned 2 records. One from 2005 and one from the current date.

Today it returns 19, no, 20 records. One from 2005, and the rest from 2020 (including this one.)

Progress.

[jnh]

Noticing both www.outbackpower.com and forum.outbackpower.com seem to be running nginx, it should be possible to set up a redirection policy on the www.outbackpower.com virtual server to recognize old-format links coming in from search engines and bounce them over to the new server, while taking /forum/ out of the URLs. Sending these as "permanent" (type 301) redirects can also help search engines update their indexes.

I don't have an nginx server on hand to test with, only Apache, but I think a line like

Code: Select all

    rewrite ^/forum/(.*) https://forum.outbackpower.com/$1 permanent;

should work, dropped into the config stanzas for www.outbackpower.com (not forum.outbackpower.com, since it needs to redirect from the old paths). Depending on the server setup this may live in /etc/nginx/nginx.conf, or maybe /etc/nginx/sites-available/www.outbackpower.com.conf or similar. There will probably be separate sections for http & https, requiring the rewrite line to be inserted into both:

Code: Select all

server {
  listen 80;
  server_name www.outbackpower.com;
 
  rewrite ^/forum/(.*) https://forum.outbackpower.com/$1 permanent;
}

server {
    listen 443 ssl http2;
    server_name www.outbackpower.com;
    ssl on;
   # expect lots of lines here for certificates, SSL options, ciphers etc.

   rewrite ^/forum/(.*) https://forum.outbackpower.com/$1 permanent;
}

Then a command like "service nginx restart" (can also vary by server type) will be needed to make it pick up the changes.

Of course you'd want to back up any config files before changing, just in case something breaks.

[David LeBow]

A) The forum search index has been rebuilt.

B) Cristian is my/our hero for unblocking/restarting the process.

Hopefully, search performance should be improved. If not... I'm a little stumped on how much more we can convince the board engine to improve its search results.

[raysun]

My litmus or should I say lithium test latest results.

This is up from the initial test of 4 results.

The whole date range is represented AFIK.

[raysun]

Sweet Mother of Jesus. Somebody needs to stage an intervention.

[raysun]

The contents of this search needs to be bound into the Holy Bible OB Editition

[raysun]

Good job guys!

Now about those external links from Google searches...