Visit Our UNIX and Linux User Community


Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations


 
Thread Tools Search this Thread
The Lounge What is on Your Mind? Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations
# 15  
Old 04-17-2020
Update:
  • Staging server is running the "two container, nginx reverse proxy to unix socket" configuration.
  • Future production server is running the "two container without nginx reverse proxy" configuration.
  • Plan to switch future production server to the "two container, nginx reverse proxy to unix socket" configuration tomorrow or Sunday (site will go down for a short time).
  • @Scrutinizer continues to make heroic progress to escape special characters outside code blocks and tags which interfere with the markdown process on discourse.

Making progress......

Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations-screen-shot-2020-04-17-40034-pmjpg
These 5 Users Gave Thanks to Neo For This Post:
# 16  
Old 04-18-2020
Update:

There is an anomaly regarding the rebuilding process (in the container) of the custom avatars in the "two container, nginx reverse proxy to unix socket" configuration.

See this post on meta:

Avatars lost after restore. How to get them back? - support - Discourse Meta

I have been working on this for a few days, and this issue only appears in the configuration of using nginx as a reverse proxy to a unix socket, but works fine without it.

So far, no resolution.
# 17  
Old 04-18-2020
Update:
  • I have successfully moved all servers over to the "two container (data, web) with nginx reverse proxy to unix socket" configuration.
  • The problem was missing information in the support tutorials, which I had to figure out on my own (again).

Basically, here is the problem:

When you follow the tutorials on meta, as I did, they don't tell you that after the transformation is done (create new data, socket-only containers); you must either:
  • Do a complete DB restore (even thought you already have a working copy of the DB in your data container) because you need all the images which are gzipped in the backup tar file, or:
  • Just copy the /shared/uploads directory over to your new container.

I did the latter (the copy method) and all the problems went away.

Kindly offered to update the tutorials over at meta, but instead of the guys thanking me, one of they guys started being "edgy" again. I guess this is the "new normal".. .blame the users who read the instructions when the instructions leave out key steps and thing break. I am glad when I went to the university our engineering professors did not blame us when the text books were wrong!

Anyway.... it's working now:

Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations-screen-shot-2020-04-18-23255-pmjpg



There are zero errors in the web dev console:

Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations-screen-shot-2020-04-18-23424-pmjpg

Last edited by hicksd8; 04-21-2020 at 09:33 AM..
These 4 Users Gave Thanks to Neo For This Post:
# 18  
Old 04-21-2020
Update:

@Scrutinizer and @Neo are getting very close to requesting our team test the new forum. We are not there yet, but getting close:
  • @Scrutinizer has written some unique custom filtering code to mitigate most of the conflicts outside code tags between special chars and markdown.
  • @Neo wrote some new code to query the legacy vB mysql db and update the discourse db to delete all "soft deleted" posts and "legacy confidential posts".

TODO:
  • Test @Scrutinzer's preprocessing code and when finished, request the team to login and test before final migration to sync new posts in old forum with new forum.
# 19  
Old 04-24-2020
Update on Migration:

There is an old Chinese saying which goes something like this:

"You cannot see the entire mountain standing in the valley".

So, we asked our mod team to do some testing and @hicksd8 immediately found that often used and abused B, I, and I BBCODE tags where causing a lot of problems embedded in other BBCODE tags.

From a statistically point of view, if we ask someone to step into a system with one million posts and within minutes, they find many of the same problems; it can easily be extrapolated that these problems are not outliers.

I discussed this B, I, U BBCODE issue with a number of people; including excellent coders, migrators, testers and casual users and after hearing a lot of opinions and personal preferences, I have made the final decision to strip out out B, I, U for the following very simple reasons:
  • We already strip out COLOR, SIZE and FONT BBCODE because these BBCODE tags also cause problems and they also make it hard for users to switch from theme to theme because COLORs that might look good in THEME A may not look good (and generally do not look good) in THEME B. This is true for SIZE, FONT and COLOR for sure. It's better to strip them out (and we did strip them weeks ago).
  • COLOR, SIZE and FONT BBCODE do not convey much meaningful information anyway, so stripping them out does not cause the forums to lose information value. In fact, in most cases COLOR, SIZE and FONT BBCODE are value subtracted and distracting.
  • The same is true, in my view (and the view of some others, but not all) for B, I, U BBCODE. This BBCODE tags do not convey much meaningful information, so stripping them out does not cause the forums to lose information value. In fact, in most cases B,I and U are value-subtracted (in my view) and distracting because users add them "based on a personal feelings and emphasis". Removing this personal emphasis does not cause the forums to lose any degree of valuable information. In fact, posts look clearer without overdone B,I and U BBCODE use and abuse.
  • Moreover, embedded B, I, U BBCODE break things in the BBCODE to MARKDOWN transformation.

I asked @hicksd8 to circle back and to do another deep "outside the mountain" view of the migration, and his comments were:

The presentation is superb and virtually nothing to comment on.

So, it's done. I have made the decision to strip out all B, I, U BBCODE and it's a done deal.

FYI, I also stripped out all MOD tags as well; because having MOD comments about "please use CODE tags" or "please show your work" over and over, is not useful information, really. Those informational comments were meant for the poster at the time, not for the ages.

@hicksd8 did have this comments:

My only concerns are regarding links. I've been checking out some of my specialist information threads (well I would wouldn't I?) first and notice that when I refer to previous threads I've written, it links to the old forum.

Let me address this:

Regarding links back to the old site:

These links from the new site to the old site will stay as they are (for the rest of this year at least) for many reasons:
  1. The old site is still live and will be live for the foreseeable future in read-only mode (as a reference site).
  2. Many, many sites with age have links referring to different domains and subdomains, this it not a "show stopper" to go back and forth. In fact, is it pretty standard for any large web site which has undergone a transformation.
  3. It is a monster of a job to rewrite all internal permalink links to match the permalink format of the new site, I am not going to spend time doing it this in 2020.
  4. There may (or may not be) be SEO value, in the short term, to have these links in the new site back pointing back to the old site.
  5. In short, am not worried about working links back to the old site, in the least in 2020, as long as they work and don't break the site, or SEO.


As @hicksd8 commented, and many agree who have recently viewed the migration:

  • The presentation is superb and virtually nothing to comment on. - hicksd8
  • AWESOME. - RavinderSingh13
  • It's 99.99% good. Apart from my concerns about intra-links, I don't think the other stuff is worth worrying about. - hicksd8

@Scrutinizer has been a tremendous help writing Ruby snippets to help clean up BBCODE to MARKDOWN issues. His excellent work is a gift to the community and will always be embedded in the migrated posts.

Soon, I will ask for comments from MadeInGermany, RudiC and other Leaders (status in the new forums).

Until then, have a great weekend.

Hope you find these updates useful.

PS: LOL... my I and B code tags (adding personal emphasis) will be stripped out in the new forums, but I'm OK with that. It's a small sacrifice for the greater good of the overall migration integrity.


Update:

Continue Here:

Please Help Integrity Test New Discourse Forums V2
These 2 Users Gave Thanks to Neo For This Post:
# 20  
Old 04-26-2020
Update1:

Update2:
  • Added two columns to the vb3 post table, discourse_topic_id and the discourse_post_id
  • Added one column to the vb3 thread table, discourse_topic_id
  • Added Ruby code to the migration script preprocessing control which inserts the discourse_topic_id and the discourse_post_id of the migrated posts back into the mysql dump for the vb3 post and thread tables.

After migration will update the original vb3 post and thread table so we have the mappings to the discourse_post_id and discourse_thread_id, which we will use for a number of redirections and links in the future.

This means, for example, when someone clicks on an old thread in the legacy vb3 forum to reply, we can send them to the new thread to reply.

Cheers.

Previous Thread | Next Thread
Test Your Knowledge in Computers #8
Difficulty: Easy
macOS is based on the Unix operating system.
True or False?

6 More Discussions You Might Find Interesting

1. What is on Your Mind?

VBulletin 3.8 to Discourse on Docker Migration Test Take Four

Test Build 4 on New Server, with changes identified in discourse test builds 2 and 3, primarily: Insuring ruby-bbcode-to-markdown is enabled. Removing line breaks from ICODE to markdown in migration script. Added vbpostid to posts in discourse to setup migrating vb "thanks" to discourse... (28 Replies)
Discussion started by: Neo
28 Replies

2. What is on Your Mind?

VBulletin 3.8 to Discourse on Docker Migration Test Take Two

OK. Like we all do, we learn a lot from tests, test migrations, and so forth. Today, I started from scratch on test migration 2, armed with a lot more knowledge, The main differences are as follows: Installed discourse plugin ruby-bbcode-to-md before starting the install Modified... (30 Replies)
Discussion started by: Neo
30 Replies

3. What is on Your Mind?

Status of Migration of Moderation Systems

First a bit of history .... A number of years ago one of our admins built a number of plugin systems for moderation, including (1) a voting system, (2) a "user feelings" system and (3) a confidential posting system. During this time, I was busy on other projects, not very active in the forums,... (1 Reply)
Discussion started by: Neo
1 Replies

4. Programming

How to track table status delete/update/insert status in DB2 V10 z/os?

Dear Team I am using DB2 v10 z/os database . Need expert guidance to figure out best way to track table activities ( Ex Delete, Insert,Update ) Scenario We have a table which is critical and many developer/testing team access on daily basis . We had instance where some deleted... (1 Reply)
Discussion started by: Perlbaby
1 Replies

5. HP-UX

Migration - Compiler Issues.

All, We are migrating an application from HP-UX B.11.00 to HP-UX B.11.31 and both of them have the same informix version - 7.25se. However the compilers are different on both servers. HP-UX B.11.00 - has B3913DB C.03.33 HP aC++ Compiler (S800) HP-UX B.11.31 - has PHSS_40631 1.0 HP C/aC++... (2 Replies)
Discussion started by: helper
2 Replies

6. Shell Programming and Scripting

Shell Script migration issues

Hi All, We will be doing a Solaris 8 to Solaris 10 migration migration, just wanted to know if there are any known / common issues arise from this migration from Shell script point of view. I tried searching this site but mostly post are related to SA's question and jumpstart, etc. If there's... (4 Replies)
Discussion started by: arvindcgi
4 Replies

Featured Tech Videos