What is on Your Mind?

04-18-2020

Administrator

19,118, 3,359

Join Date: Sep 2000

Last Activity: 15 July 2022, 8:51 AM EDT

Location: Asia Pacific, Cyberspace, in the Dark Dystopia

Posts: 19,118

Thanks Given: 2,351

Thanked 3,359 Times in 1,878 Posts

Update:

There is an anomaly regarding the rebuilding process (in the container) of the custom avatars in the "two container, nginx reverse proxy to unix socket" configuration.

See this post on meta:

Avatars lost after restore. How to get them back? - support - Discourse Meta

I have been working on this for a few days, and this issue only appears in the configuration of using nginx as a reverse proxy to a unix socket, but works fine without it.

So far, no resolution.

Neo

04-18-2020

Administrator

19,118, 3,359

Join Date: Sep 2000

Last Activity: 15 July 2022, 8:51 AM EDT

Location: Asia Pacific, Cyberspace, in the Dark Dystopia

Posts: 19,118

Thanks Given: 2,351

Thanked 3,359 Times in 1,878 Posts

Update:

I have successfully moved all servers over to the "two container (data, web) with nginx reverse proxy to unix socket" configuration.
The problem was missing information in the support tutorials, which I had to figure out on my own (again).

Basically, here is the problem:

When you follow the tutorials on meta, as I did, they don't tell you that after the transformation is done (create new data, socket-only containers); you must either:

Do a complete DB restore (even thought you already have a working copy of the DB in your data container) because you need all the images which are gzipped in the backup tar file, or:
Just copy the /shared/uploads directory over to your new container.

I did the latter (the copy method) and all the problems went away.

Kindly offered to update the tutorials over at meta, but instead of the guys thanking me, one of they guys started being "edgy" again. I guess this is the "new normal".. .blame the users who read the instructions when the instructions leave out key steps and thing break. I am glad when I went to the university our engineering professors did not blame us when the text books were wrong!

Anyway.... it's working now:

Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations-screen-shot-2020-04-18-23255-pmjpg

There are zero errors in the web dev console:

Update on vB3 Migration to Discourse - Issues and Status of BBCode Transformations-screen-shot-2020-04-18-23424-pmjpg

Last edited by hicksd8; 04-21-2020 at 09:33 AM..

These 4 Users Gave Thanks to Neo For This Post:

Neo

04-21-2020

Administrator

19,118, 3,359

Join Date: Sep 2000

Last Activity: 15 July 2022, 8:51 AM EDT

Location: Asia Pacific, Cyberspace, in the Dark Dystopia

Posts: 19,118

Thanks Given: 2,351

Thanked 3,359 Times in 1,878 Posts

Update:

@Scrutinizer and @Neo are getting very close to requesting our team test the new forum. We are not there yet, but getting close:

@Scrutinizer has written some unique custom filtering code to mitigate most of the conflicts outside code tags between special chars and markdown.
@Neo wrote some new code to query the legacy vB mysql db and update the discourse db to delete all "soft deleted" posts and "legacy confidential posts".

TODO:

Test @Scrutinzer's preprocessing code and when finished, request the team to login and test before final migration to sync new posts in old forum with new forum.

Neo

04-24-2020

Administrator

19,118, 3,359

Join Date: Sep 2000

Last Activity: 15 July 2022, 8:51 AM EDT

Location: Asia Pacific, Cyberspace, in the Dark Dystopia

Posts: 19,118

Thanks Given: 2,351

Thanked 3,359 Times in 1,878 Posts

Update on Migration:

There is an old Chinese saying which goes something like this:

"You cannot see the entire mountain standing in the valley".

So, we asked our mod team to do some testing and @hicksd8 immediately found that often used and abused B, I, and I BBCODE tags where causing a lot of problems embedded in other BBCODE tags.

From a statistically point of view, if we ask someone to step into a system with one million posts and within minutes, they find many of the same problems; it can easily be extrapolated that these problems are not outliers.

I discussed this B, I, U BBCODE issue with a number of people; including excellent coders, migrators, testers and casual users and after hearing a lot of opinions and personal preferences, I have made the final decision to strip out out B, I, U for the following very simple reasons:

We already strip out COLOR, SIZE and FONT BBCODE because these BBCODE tags also cause problems and they also make it hard for users to switch from theme to theme because COLORs that might look good in THEME A may not look good (and generally do not look good) in THEME B. This is true for SIZE, FONT and COLOR for sure. It's better to strip them out (and we did strip them weeks ago).
COLOR, SIZE and FONT BBCODE do not convey much meaningful information anyway, so stripping them out does not cause the forums to lose information value. In fact, in most cases COLOR, SIZE and FONT BBCODE are value subtracted and distracting.
The same is true, in my view (and the view of some others, but not all) for B, I, U BBCODE. This BBCODE tags do not convey much meaningful information, so stripping them out does not cause the forums to lose information value. In fact, in most cases B,I and U are value-subtracted (in my view) and distracting because users add them "based on a personal feelings and emphasis". Removing this personal emphasis does not cause the forums to lose any degree of valuable information. In fact, posts look clearer without overdone B,I and U BBCODE use and abuse.
Moreover, embedded B, I, U BBCODE break things in the BBCODE to MARKDOWN transformation.

I asked @hicksd8 to circle back and to do another deep "outside the mountain" view of the migration, and his comments were:

The presentation is superb and virtually nothing to comment on.

So, it's done. I have made the decision to strip out all B, I, U BBCODE and it's a done deal.

FYI, I also stripped out all MOD tags as well; because having MOD comments about "please use CODE tags" or "please show your work" over and over, is not useful information, really. Those informational comments were meant for the poster at the time, not for the ages.

@hicksd8 did have this comments:

My only concerns are regarding links. I've been checking out some of my specialist information threads (well I would wouldn't I?) first and notice that when I refer to previous threads I've written, it links to the old forum.

Let me address this:

Regarding links back to the old site:

These links from the new site to the old site will stay as they are (for the rest of this year at least) for many reasons:

The old site is still live and will be live for the foreseeable future in read-only mode (as a reference site).
Many, many sites with age have links referring to different domains and subdomains, this it not a "show stopper" to go back and forth. In fact, is it pretty standard for any large web site which has undergone a transformation.
It is a monster of a job to rewrite all internal permalink links to match the permalink format of the new site, I am not going to spend time doing it this in 2020.
There may (or may not be) be SEO value, in the short term, to have these links in the new site back pointing back to the old site.
In short, am not worried about working links back to the old site, in the least in 2020, as long as they work and don't break the site, or SEO.

As @hicksd8 commented, and many agree who have recently viewed the migration:

The presentation is superb and virtually nothing to comment on. - hicksd8
AWESOME. - RavinderSingh13
It's 99.99% good. Apart from my concerns about intra-links, I don't think the other stuff is worth worrying about. - hicksd8

@Scrutinizer has been a tremendous help writing Ruby snippets to help clean up BBCODE to MARKDOWN issues. His excellent work is a gift to the community and will always be embedded in the migrated posts.

Soon, I will ask for comments from MadeInGermany, RudiC and other Leaders (status in the new forums).

Until then, have a great weekend.

Hope you find these updates useful.

PS: LOL... my I and B code tags (adding personal emphasis) will be stripped out in the new forums, but I'm OK with that. It's a small sacrifice for the greater good of the overall migration integrity.

Update:

Continue Here:

Please Help Integrity Test New Discourse Forums V2

These 2 Users Gave Thanks to Neo For This Post:

Neo

04-26-2020

Administrator

19,118, 3,359

Join Date: Sep 2000

Last Activity: 15 July 2022, 8:51 AM EDT

Location: Asia Pacific, Cyberspace, in the Dark Dystopia

Posts: 19,118

Thanks Given: 2,351

Thanked 3,359 Times in 1,878 Posts

Update1:

Now in data integrity testing phase: Please Help Integrity Test New Discourse Forums V2

Update2:

Added two columns to the vb3 post table, discourse_topic_id and the discourse_post_id
Added one column to the vb3 thread table, discourse_topic_id
Added Ruby code to the migration script preprocessing control which inserts the discourse_topic_id and the discourse_post_id of the migrated posts back into the mysql dump for the vb3 post and thread tables.

After migration will update the original vb3 post and thread table so we have the mappings to the discourse_post_id and discourse_thread_id, which we will use for a number of redirections and links in the future.

This means, for example, when someone clicks on an old thread in the legacy vb3 forum to reply, we can send them to the new thread to reply.

Cheers.

Neo