Bitbucket set to remove Mercurial support

[kitor]

I guess this should be fixed, of course it leads to nowhere.

[DeafEyeJedi]

According to the first joke from here, the problem is solved - you may consider I'm the mathematician

Joke of the year. Great work, @a1ex!

[a1ex]

Thanks - for those who didn't get the joke, it means somebody still has to sit down and perform the work

Which is exactly the problem I'm trying to solve with Open Collective.

[PaulJBis]

Thanks - for those who didn't get the joke, it means somebody still has to sit down and perform the work

Which is exactly the problem I'm trying to solve with Open Collective.

Excuse me if I'm missing something, but if I'm not mistaken, the tasks you mentioned above (rewriting all the links in the forum to go from Bitbucket to Heptapod) seem pretty much trivial, right? It's just a matter of loading the SQL dump of this forum and doing a search/replace. (the commit IDs are a different matter, admittedly).

I am asking this because I've long wanted to give some help to this project, after using it for so many years, but I have no experience with the kind of low level programming that you guys need, so when I read this, it sounded like something so easy that even *I* could do it...

(As for migrating the commit IDs, that might be above my skill level, but also: I've been looking around in the forum and the website, but haven't been able to find any link to specific commit IDs in Bitbucket. Do you have an example? )

[a1ex]

Pretty much, yes. However, I'd prefer a "soft" replacement, when displaying the forum posts, rather than permanently altering the forum database. There's plenty of room for making non-obvious mistakes (that could be noticed months after the change).

You can find several examples by typing "bitbucket.org" in the forum search box, though for a complete list, one may need admin access. Or, after fixing the most common classes of links, the search box will reveal the remaining ones. Writing down the search/replace patterns would be a significant time saver. Caveat: it's not as simple as replacing "bitbucket.org" with "heptapod.net".

Though, the problem I've been trying to delegate to the other members of the community, is the one about migrating the commit IDs. Search string: "bitbucket.org/hudson/magic-lantern/commits", but there are also links from other repositories, that should be handled somehow. Heptapod only hosts the main repo, without the forks, but I've saved an archive of all of the forks before they got deleted - link earlier in this thread. Forks were also archived by softwareheritage.org (helped by the team behind Heptapod), as mentioned here, so linking to them could be an option as well. Again, last time I've checked, the commit IDs didn't match.

[PaulJBis]

Okay, let's separate this into two problems: the "normal" links to Bitbucket and the links to specific commit IDs.

Pretty much, yes. However, I'd prefer a "soft" replacement, when displaying the forum posts, rather than permanently altering the forum database. There's plenty of room for making non-obvious mistakes (that could be noticed months after the change).

If you don't want to touch the forum DB itself, the solution would be to add a HTTP redirection somewhere. But the problem is that, since all the links still point to the bitbucket.org domain, either you convince the Bitbucket guys to add that redirection for you, or you'll still have to touch the forum posts, if only to point them to your own redirector.

What I mean is: you'll have to edit all the links so that they point to "myownbitbucket.org" (for example),  and add in that domain a server that would take into account all the different paths and redirect the user transparently to the appropiate URL. We'd still be editing the forum posts, but only to change the domain part (from bitbucket.org to myownbitbucket.org or whatever); the rest of the URL, which is where the "danger" lies, would be rewritten in the new redirect server. Does that sound good to you, or is it too convoluted? (I could handle the redirect server).

As for the commit IDs, my impression after reading this thread is that the simplest solution would be to ask the Heptapod people for a copy of the git-mapfile. Am I mistaken, or is that a static file that doesn't change anymore? (I mean: the new commits are done directly into the Heptapod server, right? They don't have any corresponding commits in Bitbucket). Either that, or reverse engineer how Heptapod generates the new commit IDs...

[a1ex]

This could actually work - and it doesn't have to be on a different domain. Replacing "^https://bitbucket.org/" with something like "^https://magiclantern.fm/bitbucket.php?u=" could do the trick. This replacement can be done on the fly in the forum code, without modifying the database, so if you'd like to test things temporarily on a different host, that's also possible.

Regarding the commit IDs, I also prefer asking Heptapod for a copy of their the git-mapfile. Still need to find the mental energy for doing that...

BTW, if you prefer to discuss the details on IRC or Discord, I'm keeping an eye of these channels as well.

[PaulJBis]

Wow, I haven't been on IRC in many years, and I still have to figure out Discord. I don't know if I'm a dinosaur or not enough of one... 

So we would have to hack the forum code then, I guess. Should I just download the code from Simple Machines and start hacking at it, or do you guys have installed any custom mods in this forum? I can also start making a "map" of URLs from the old repo and their correspondences in the new one, as time permits.

(If you need to discuss any details privately, just send me a message here for the moment, until I've figured out Discord...)

[PaulJBis]

Well, I'v been having a look to all the bitbucket.org links I could find in a quick search, and I saw the following patterns:

• Branches. Those links have the format https://bitbucket.org/hudson/magic-lantern/commits/branch/(branch name), and they translate to https://foss.heptapod.net/magic-lantern/magic-lantern/-/tree/branch/(branch name).
  
• Issues in the bug tracker. They have the format https://bitbucket.org/hudson/magic-lantern/issues/(issue number)/(title of the issue), and they translate to https://foss.heptapod.net/magic-lantern/magic-lantern/-/issues/(issue number)
  
• Commit IDs. They have the format https://bitbucket.org/hudson/magic-lantern/commits/(commit ID). We'll leave these for later, as we need the git-mapfile.
  
• Pointers to a file in a given commit. For example: https://bitbucket.org/hudson/magic-lantern/src/d318f774b24ecc7d533ab79dd9418d097442fd81/src/raw.c#lines-905. We'll leave these for later too.
  

And now there are some URLs where I don't know very well what to do with them:

• Links to a given file. In Bitbucket they were like this: https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c (for example). The problem is that the Heptapod link includes the branch information in it: https://foss.heptapod.net/magic-lantern/magic-lantern/-/blob/branch/unified/modules/dual_iso/cr2hdr.c, so I don't know where to get it from in the Bitbucket links.
  
• Links from repos created by other developers. For example: https://bitbucket.org/Dannephoto/switch_mini-git/src/master/, http://bitbucket.org/coutts/1000d_dev/src/default/, etc. Have they all been ported to Heptapod?
  

[kitor]

Links to a given file. In Bitbucket they were like this: https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c (for example). The problem is that the Heptapod link includes the branch information in it: https://foss.heptapod.net/magic-lantern/magic-lantern/-/blob/branch/unified/modules/dual_iso/cr2hdr.c, so I don't know where to get it from in the Bitbucket links.

well, there's "tip" so there's a tag in URL. Not sure how to map this to heptapod though.

Tip is the most recent changeset in the repository. Default is a branch. Tip is a special Tag (a named changeset) which always refers to the most recently changed Head.

Which also means that URLs with "tip" are... useless

[a1ex]

Probably the most reasonable way to interpret that "tip" would be "tip at the time of writing". That is, link translation should be done considering the date of the post containing that link.

Otherwise, replacing it with "unified" could also work.

For the forks, which are not hosted on Heptapod, we can probably link to Software Heritage. Example: https://archive.softwareheritage.org/browse/origin/content/?origin_url=https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0&path=src/video_hacks2.c

Or, I could merge all of the forks into one big repo (let's say magic-lantern-all-forks) and ask Heptapod nicely to create it and let me push everything there. I don't have permission to create new repos there, but I've been thinking to ask them to re-host some of the "side projects" related to ML, such as ml-focus-pixels etc, in the same namespace.

[PaulJBis]

Probably the most reasonable way to interpret that "tip" would be "tip at the time of writing". That is, link translation should be done considering the date of the post containing that link.

Otherwise, replacing it with "unified" could also work.

The problem I see with those links is the lack of branch information. I mean, with a link like  https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c, how do we know which branch it's pointing at? Can we safely assume it's for the default branch "unified"?

[kitor]

Honestly, without knowing author thoughts at the time of writing specific post, you never know.

Probably the most reasonable way to interpret that "tip" would be "tip at the time of writing". That is, link translation should be done considering the date of the post containing that link.

Alex's idea seems right. If this was routed via internal "proxy" then redirect page might warn user that this points to unknown branch. Requires additional coding though.

[Audionut]

Can we safely assume it's for the default branch "unified"?

My recollection is that a branch always had the branch name in the link, so yes, it would be safe to assume that IMO.

Otherwise, replacing it with "unified" could also work.

[natschil]

I'm currently unable to get the source code from Heptapod. I have no experience with mercurial whatsovever, but I tried "hg clone https://foss.heptapod.net/magic-lantern/magic-lantern". This gave me some source code that is different to what I see on Heptapod. I also tried the mercurial to git approach suggested here, that gave me what seems like the same source code as before, here the last commit is one from 2011. It would be helpful to have a file that gives you instructions on how to install the source code somewhere.


Edit: it seems like doing "hg checkout unified" does the trick.