Bitbucket set to remove Mercurial support

[names_are_hard]

I think this affects ML.  Maybe not, they talk about Bitbucket Cloud so perhaps we're on a service where they're not retiring Mercurial?

https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket
"Mercurial features and repositories will be officially removed from Bitbucket and its API on June 1, 2020"

I don't like their decision to delete repos.  Putting them in a read-only mode would have been a lot kinder.

There are tools to migrate to Git (so I guess you keep your history?) but I know from personal experience that building ML has dependencies on having hg installed on your system.  It wasn't hard to remove these.

Long term this is probably good for ML - almost no-one uses Mercurial and needing to learn it must put some people off ML.  Short term it's annoying to migrate!

[dfort]

Yep, just got the email from Atlassian.

After much consideration, we've decided to remove Mercurial support from Bitbucket Cloud and the API. Mercurial features and repositories will be officially removed from Bitbucket and its API on June 1, 2020.

What used to be a very fragmented version control software market has rapidly matured. Mercurial usage on Bitbucket is steadily declining, and the percentage of new Bitbucket users choosing Mercurial has fallen to less than 1%. At the same time, Git has become the standard. According to a Stack Overflow Developer Survey, almost 90% of developers use Git, while Mercurial is the least popular version control system with only about 3% developer adoption.

Given that Git has emerged as the version control system of choice for the industry, we have decided to focus our priorities and roadmap on building the best possible experience for the market and our users.

Read on to learn more about what led to this decision, timeline, and migration resource support.

[Danne]

Aw, hell...

[names_are_hard]

Meh, annoying.  Could also move to another provider that supports Mercurial, I suppose.  Not my call.

[a1ex]

Aw, hell...

Indeed, unlearning ~ 10 years worth of workflow is not going to be funny

Not only that, but there's a lot of useful info on Bitbucket (discussions on issue tracker, pull requests, various forks) that will be deleted, even if we move to some other Mercurial host (correct me if I'm wrong). These will have to be migrated (manually copy/pasted?) somewhere, too. How? Maybe web.archive.org to the rescue?

Migrating to git also means the existing commit hashes (used all over the place in documentation and discussions) will no longer work. Updating those is going to be a PITA, just like the Dropbox links used a lot in the past (still unsolved as of today).

I actually had a few attempts at learning Git, but found it a lot more complicated than Mercurial. I'm not completely against switching to Git, but I need some very good reasons to do so.

"ML has dependencies on having hg installed" -> this one is likely easy to fix. Still a PITA, will take at least a couple of hours, but doable.

In any case, no matter what path we'll choose, this change will result in lots of time wasted (time that would have otherwise been spent on development), and plenty of headaches.

I actually hope there will be some miracle at Bitbucket, about reversing this decision...


Discussions across the web:
- https://community.atlassian.com/t5/Bitbucket-articles/What-to-do-with-your-Mercurial-repos-when-Bitbucket-sunsets/ba-p/1155380
- https://www.reddit.com/r/programming/comments/csy2tf/bitbucket_kills_mercurial_support/
- https://news.ycombinator.com/item?id=20745393

[nikfreak]

'nuff time til June May 2020. I am sure there will be a way to either archive everything (read-only mode) or migrate almost everything.

[Danne]

'nuff time til June May 2020. I am sure there will be a way to either archive everything (read-only mode) or migrate almost everything.

Well it's not gonna migrate overnight, that's for sure. Tragic. Personally I built so many tools around mercurial and now it feels like a strong grown forest and it's source trees, branches, the works is just killed.
And then git, sigh...
I guess nothing lasts forever. Except for maybe eos m 

[names_are_hard]

Finding the best compromise between least painful and most information preserved may be time consuming though

I am trying Github's import function, apparently that is quite reliable.  I'll update when it's finished with what things worked well / badly.  In theory one option would be hg-git, allowing Github (or Bitbucket, but you'd still need to migrate the repo, for which Github has a function already and Bitbucket doesn't...) for hosting, and devs can use hg or git cli tools:
https://hg-git.github.io/

In the community thread, Atlassian staff seem quite convinced they won't change their minds.  They even rule out developing an auto-migration tool for your repo type on Bitbucket.  Sometimes angry internet people get stupid decisions changed, so we can wait and see.

Personally I prefer Git...  but really only because I've never learnt Mercurial.  Reading around it seems they have basically identical functionality these days, just accessed in different ways.  But they are both complicated tools and relearning is pain, so keeping the ability to use hg tools seems valuable.

[dmilligan]

Well, hg always annoyed me b/c I use git at work and I am more used to it, but this move by atlassian is pretty sketchy. I’d see this as an opportunity to dump them altogether and moving to another hosting provider. Although as a1ex points out it does suck to loose all that history. Maybe they will relent as google did with nest recently. We should all probably complain loudly, just in hopes of keeping options open.

[names_are_hard]

Attempt one to import to Github failed, saying there were import errors, but no description beyond that.  Not very useful...

I'll give it another try and if that fails probably mess around with the Atlassian recommended CLI tools for import / export.

[names_are_hard]

Using hg-fast-export I was able to get this to work.  This also had a problem (maybe the same as Github), because Mercurial allows branch names with a leading "/" and Git doesn't.  You need to specify a branch mapping file, I used one with the contents like this:

"/raw_recc-added-121-and-11751-aspect-rati-1382109783386"="raw_recc-added-121-and-11751-aspect-rati-1382109783386"
"/added-1792-resolution-1390499711758"="added-1792-resolution-1390499711758"

Then it worked fine.  In theory it has preserved the hg hashes as I chose the option to "Annotate commits with the hg hash as git notes in the hg namespace".  Haven't tested thoroughly, but you can get what look like useful mappings of Git commits to hg hashes via: git notes --ref hg list

I then did: git push --all
This may not strictly be needed, but I don't understand the branch model of Mercurial so wanted to save all of them.  There are 274 open branches???

Anyway, the result is here:
https://github.com/reticulatedpines/magiclantern/tree/unified

Overall, pretty easy, and the results look sensible to me.  But I'm not qualified to test hg tools against that repo using hg-git.  If anyone competent is interested in testing that, let me know if you run into any problems.  It does show that you can take all the history out at least.  The same procedure should work for moving from hg to git while remaining on Bitbucket, too.

I've also found a script for moving Bitbucket issues to Github, I'm trying that out.

[names_are_hard]

The issue migration is not horrible, could use a little work.  Here's an example:
https://github.com/reticulatedpines/magiclantern/issues/2

One obvious ugly part, it thinks I'm the reporter for all issues.  It's also using a dumb heuristic to link the Bitbucket account of people in the tickets to a Github account.

More subtle, while it preserves the links to the hg commits, these link to Bitbucket so will shortly become useless.  This is fixable, I would need to modify the migration script to understand the hash mappings made by hg-fast-export, probably pretty easy, I can see where in the code this would happen.

So, fairly hopeful if the Git route is chosen!  We can automatically keep the commit history of all branches, we can keep all the issues, and we can map the hg commits to git commits (probably including doing this automatically for issues at least).

[kitor]

Then it worked fine.  In theory it has preserved the hg hashes as I chose the option to "Annotate commits with the hg hash as git notes in the hg namespace".  Haven't tested thoroughly, but you can get what look like useful mappings of Git commits to hg hashes via: git notes --ref hg list

Yup, that should work. We are migrating SVN workflow to Git and protocol v1 gets overwhelmed if you have too many tags in your repo (and we are creating a ton of them every day). Git notes was found to be one of better solutions for this problem - you can search them easily.

[chris_overseas]

This might be an easier option?

https://sourcehut.org/blog/2019-08-21-sourcehut-welcomes-bitbucket-refugees/

[names_are_hard]

Some kind of Mercurial hosting is probably the better option - the main devs apparently prefer Mercurial, plus the migration may well be easier.  I do note this from Sourcehut, https://sourcehut.org/alpha-details/ :

Payment is optional now but will be required later

I was testing the Git side just for feasibility   Well, that and I am using Git for my fork because I have zero Mercurial experience, but that shouldn't factor in to ML future choices.  (If I get enough progress for my fork to be useful, I will merge those change back into Mercurial ML - hopefully without hg eating my files.  Again.)

[Luther]

This might be an easier option?

https://sourcehut.org/blog/2019-08-21-sourcehut-welcomes-bitbucket-refugees/

@chris_overseas is right. Sourcehut is awesome. You don't need to pay, you can self host too. If git is too complicated, see also Pijul and Game of Trees