Similar Threads: More UNIX and Linux Forum Topics You Might Find Helpful Update


 
Thread Tools Search this Thread
The Lounge What is on Your Mind? Similar Threads: More UNIX and Linux Forum Topics You Might Find Helpful Update
# 1  
Old 08-12-2018
Similar Threads: More UNIX and Linux Forum Topics You Might Find Helpful Update

Today I change the DB and the PHP code and rebuilt the database for similar threads at the end of each post, increasing from a max of 5 to a max of 10 similar threads per post:

More UNIX and Linux Forum Topics You Might Find Helpful

It was quite easy to do:

1. Increased the max size of the similar field in the thread DB from 55 to a 120 varchar.

2. Changed a fetch_similar_threads($threadtitle, $threadid = 0) DB PHP query from LIMIT 5 to LIMIT 10.

3. Rebuild the entire thread table of all (currently 280K) threads.

Done!
This User Gave Thanks to Neo For This Post:
# 2  
Old 08-12-2018
Brilliant, thank you.
I found an overbalance of the proposals lying in the past, omitting / ignoring recent contributions. Not implying the recent ones are better, but would it be feasible to make distribution of selected threads spread somewhat more evenly?
This User Gave Thanks to RudiC For This Post:
# 3  
Old 08-12-2018
When the DB is rebuilt those issues are taken care of.

Moving forward I will rebuild monthly.
This User Gave Thanks to Neo For This Post:
# 4  
Old 08-13-2018
This issue is that these "Similar Threads" are built automatically when new threads are created.

The code runs thru the database, scores similar threads primarily based on the title of the threads, sorts the threads based on the scores, and then places a comma separated field of the thread ids in each thread in a DB field for that thread. Before that number was limited to 5 and now it is limited to 10. I may increase if to 15 or even 20 people think it is a good idea.

However, there was a number of logic flaws in the process. The main flaw is that old threads are not updated with "similar ones" unless we manually update the database. This was not done on a regular basis since it was designed to be done manually; but I have it on my calendar to run it once a month.

The second flaw is the algorithm used to create the matches is based mostly on thread titles. This is not bad, but it would be better if it used the thread content or at least the tags. However, this of course will effect performance.

If you are interested, I can post the entire PHP code for this algorithm and you can rewrite it to make it even better. Otherwise, it will be on a future "Neo TODO List" as I'm now focused on tasks related to the layout, tables and responsive design". I can only do so much and juggle so many balls at once, LOL

On my current main TODO list are:

  • Responsive threadbits (search results, forum thread views, etc.).
  • Responsive similar thread (under each thread).

Each of these tasks is fairly simple, since I have done this a few time already:

  • Study the source of a page and determine where to add CSS flex container classes.
  • Add CSS flex container classes to the rows and row items.
  • Write jQuery code to manipulate the DOM for these classes.
  • Test this in "Neo Only Mode" until I get all the parameters working correctly.
  • Check responsiveness from huge (2000px wide) to small (300 px wide) and retest.
  • Make public after "Neo Testing" is done.

It was interesting to do at first, but the more tables I transform to CSS flex, the more boring it becomes, so occasionally I diverge into fixing something else just to keep my sanity, LOL

For example, a few days ago I got bored and decided to add some new "tooltips" to the thread bits using Bootstrap and their tooltip extensions. I need to add this to all threads and forum descriptions and then create some UserCP bools or client side JS code to turn this on and off. I was going to code this inside the browser (Javascript) and not in the UserCP (PHP, server side), but then I will need to use a cookie to "remember" the user selections.

If I do this now (create the cookies), I will end up with 20 more cookies, which I do not want to do, so I want to wait until I have most of the new .JS done, create a single cookie with a JSON configuration string and encrypt this string in the cookie. This is all doable, but I prefer to do this after finishing 80% of the responsible jQuery code I need to write.

In the past, we had up to 3 "coding" admins, or at least 2, and now we are down to me, which is fine. I greatly appreciate all the coding in the past, but as the only "coding guy" these days, I have to follow some logical plan to get this done, while I have the time and interest.

Cheers.
# 5  
Old 08-13-2018
Unfortunately, I have absolutely no command of PHP, so posting the code would not fall on fertile soil. That's also why I regrettably can't offer any help in the project.
# 6  
Old 08-13-2018
If I see a thread with a title containing a misspelled word (e.g. the recent thread "Troubleshoting whiptail"), should I correct the misspelled word or should I leave it as it was submitted? Will changing a word in a title screw up finding "similar threads" or will correcting it make it more likely that "similar threads" will be found?
# 7  
Old 08-13-2018
I always correct spelling errors in works when I see them.

Yes, if there is a spelling error and the similar threads have already been created, the similar thread entry for that post will need to be rebuilt to get a more accurate match.

If there is a spelling error in the title, and the similar thread entry is created / calculated, then of course it will have poor matches. Changing the spelling is good, but the entry for similar threads in the DB entry for that thread will be "off" until the DB is rebuilt.

So best to always correct spelling, and then later when the DB is updated (similar threads rebuilt) the new matches will be much better.
This User Gave Thanks to Neo For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

4 More Discussions You Might Find Interesting

1. What is on Your Mind?

Similar Threads for Man Pages - In Development

FYI, I have been quietly updating the man page database adding "similar threads" for man pages. STEP 1: Full Text MySQL DB Search Matches The first step, after creating the DB columns, was to process each of the nearly 400K man pages and do a full text mysql search, match and score... (10 Replies)
Discussion started by: Neo
10 Replies

2. What is on Your Mind?

Similar Threads Redesign for UNIX.com

Hello. I have redesigned our "similar threads" for a more "clean style" in both the desktop view and the mobile view for the forum "show thread". The new design is a simple straight-forward use of div elements and all the legacy table elements have been removed. The result is a "clean"... (7 Replies)
Discussion started by: Neo
7 Replies

3. What is on Your Mind?

Forum Update New Steps for UNIX.com

Dear Everyone, Thank you for all the great comments, feedback and patience during our recent modernization efforts at unix.com. Now, I need to decide where to go next, as we move into the next phase. Some of the ideas I have are: Make the UserCP experience and all those non-public... (6 Replies)
Discussion started by: Neo
6 Replies

4. News, Links, Events and Announcements

Similar Threads - a new vB3 feature for UNIX.COM

Note the new feature on UNIX.COM, many thread at the bottom of the page have a new feature: Similiar Threads Here is an example on one of Perderabo's posts: https://www.unix.com/showthread.php?t=16337 Kudos to the vB folks for this built-in feature!! Neo (0 Replies)
Discussion started by: Neo
0 Replies
Login or Register to Ask a Question