Bazaar Explorer Branches

alanjfpaterson

I mentioned on the chat window last week that I had started thinking about the way we use Bazaar. As you know, Bazaar is an example of Distributed Version Control - as opposed to Centralised Version Control (like SubVersion). There is loads of discussion on the web about the relative merits. Centralised control was the original topology, in which a central repository held the trunk or a branch and clients checked out files (only the ones that were needed), modified them and checked them back in - then along came distributed version control where the big 'advantage' was that every developer had their own branch repository (including all the files and all the history), so potentially every developer 'knew' everything about the central repository and could work offline not needing a connection to the central server.

As you will realise, this is not the way we use Bazaar. We create a branch on the server and checkout a copy of those files to our local PC. So, although we get a complete copy of all files we do not have any information about the branch - that is still all held on the server. This makes it impossible for us to collaborate on a branch (not that we have particularly needed to up to now, because our branches have a very small focus). But it would not be possible for two of us to check out from the same server branch. And whenever we want to interact with the branch - we have to do it over the internet.

I remember when I started with OpenPetra being surprised that we did not seem to get any of the reported benefits of Distributed Version Control. I had been interested to see this because the control system we used at MKS was a centralised system - but the way we use Bazaar is in fact closer to a centralised system than a distributed one! I think I was told that we had tried to set up the distributed way but it had not worked - so we do what we do.

So over the weekend I thought I would re-visit the topic. And the results have changed my life!!! You would not believe the transformation in the way that Bazaar works and how snappy it all is if you get it set up the way that it has really been designed to work. The point is that everyone who uses Bazaar should work in their own branch - which is a child of another branch. And because it is a branch (as opposed to a checkout) everyone knows everything about all the revisions. Thus, if you want to look at the history (log) - its all on your PC. If you want to look at differences - its all on your PC. If two of us want to collaborate on something, we each have a branch but we can both 'push' our revisions to our parent branch or 'pull' each others revisions from that branch to our own.

Before I explain how to achieve this (and you try to see the wonderful improvement that is possible) I need to alert you to something that you must do first. Because it did not work first time for me either and is probably why it was not possible to make it work when we tried before..... You need to set this up in a new 'root' folder and not use the same root folder that you have been using for checkouts. I think this is because Bazaar uses a lot of hidden files and folders and the ones that are in your checkout folders now will get the new paradigm confused. So I have used up to now a folder C:/OM_Source for all my dev_ checkouts. For this work I created a new folder C:/OM_Branches which obviously starts off completely empty. Please make sure you do this before you start.

You also will find the help documentation useful - at doc.bazaar.canonical.com/explore ... ndows.html - not so much because it is comprehensive but just because it does show the screens you should be seeing and the screen examples clearly assume that branches are on local C drives.

So here goes.

Start by creating a branch on the server (exactly as you do now). This will be the branch that ultimately will get merged into trunk. But this could in the future be a higher level feature branch that two of us will work together to deliver. Let us say it is some finance feature with a number of sub-features that make the whole thing up, that Chris will own it and Wolfgang will be part of the team working on it with him. So we do Bazaar | Start | Branch. The source is lp:/openpetraorg and the branch is lp:/chris-thomas/openpetraorg/dev_0009999_new_finance_feature. Click OK and after about 5 seconds the job is done because the new branch is inside the existing repository and nothing really needs creating - just a bunch of empty links. As I say - this is EXACTLY the same as we already do.
Now comes the big difference. We need to create a branch from that branch - in fact Chris and Wolfgang both do this step. Once again do Bazaar | Start | Branch. This time the 'From'is lp:/chris-thomas/openpetraorg/dev_0009999_new_finance_feature and the 'To' is C:/OM_Branches/dev_0009999_new_finance_feature. Click OK. (Note that Bazaar pretty much expected to make the branch on your hard drive.)
Because this is the first time you have created a branch on your PC you will get a dialog popping up asking if you want to create a repository. (Look at the 'Branch' section of the help document - it explains that you can expect this.) Choose the default 'Yes' and accept all the default settings on the next dialog (Shared repository, v2.0 etc). Now Bazaar starts what is a long process because our central repository contains such a long history. My PC had to acquire information about 257,000 revisions that involved over 300MB of data traffic over the internet and at my connection speed took 24 minutes! But it is worth it because my PC now has all that information locally about everything that has happened to OpenPetra.
Finally at the end of this process and without you needing to do any further action, Bazaar gets the files and puts them into a 'trunk' inside your copy of the repository. This takes no more than 10 seconds to do the 3000-odd files. Do not confuse this trunk with our trunk on launchpad - they are not the same.

And that is it. Now Bazaar has given you all the benefits of distributed version control. The usual window will probably be open saying you are up to date at revision NNNN. Try clicking the Log button. Wow! Instant! Select a check-in and pick a file in the file list window and choose to see its differences. Instant again! Hand edit a file in Notepad++ (I just picked a manual file and added a //comment and saved it). The Bazaar window updates to show that I have a modified file. Click on the difference icon - instant! Check in the change - really quick! But where did it get checked in to? Yes it got checked in to your branch on your PC. None of these actions used an internet connection. That is why they are so quick.

Before I explain what else you need to know to upload your branch to launchpad I want to mention a couple of new features that you will get from Bazaar that you have not even seen up to now (because you have been working with checkouts not branches). Have a look at the help document - because now you can take advantage of 'Repository View' (early on in help). Your local branch is a complete repository (and your code is in trunk). But this repository is linked to a parent branch (in this case lp:/chris-thomas/openpetraorg/dev_0009999_new_finance_feature). Repository view can show you the recent check-ins to your branch and can tell you that there are changes in your branch that have not been replicated to the parent branch - and also if there have been any changes in the parent that you have not brought in to your code. Here is collaboration in action. In my example, Chris needs to get his changes up to the server where Wolfgang can get at them - and vice versa, Chris needs to get Wolfgang's contributions into his branch. The 'Repository View' is what helps you here. Repositories are indicated by the brown icon and double clicking on the brown icon in the welcome screen activates repository view - or right click in Windows Explorer on the folder and choose 'Bazaar Explorer'.

Note that the repository view is telling you that you have now got revision(s) that you have committed that need to be sent to the parent branch.

OK - so what is involved in 'pushing' to or 'pulling' from the server. Now that you have edited a file and committed it to your branch, you can 'Push' all the revisions that you have committed to the parent branch. The Push dialog simply wants to know the location of the parent branch (I think it already has it filled in to the dialog). Click OK. You can go to code.launchpad.net with your browser and check that your change has made it ok, complete with your check-in comment. And that is it really - until we actually start team collaboration we won't need to 'Pull' anyone else's changes into our personal branch - but repository view would tell us if there are any changes to pull.

The only other thing is to Merge changes from trunk into our branch. That is exactly the same as we do now - but this time we are merging changes into a branch rather than a checkout - and I have a sneaky suspicion that the latest version of Bazaar would do that successfully because we would be using Bazaar in the way that Canonical would use it, so if there were bugs in that they would get fixed very quickly. (Once the revisions have been pulled from trunk to your branch they need committing to your branch and then 'Pushing' to the server branch.

I will certainly use branches rather than checkouts from now on. The usability difference is dramatic - as well as the functionality improvements. I would recommend that you do what I have done and just make a new branch to 'play' with. Nothing needs to change with the server setup - but please do set up a totally separate root folder (in C: or MyDocuments but outside any existing folders you use for Bazaar). It takes a longer time to create a branch than a checkout - but that time is more than recovered when you come to do your work. And from what I can gather my 24 minutes may only be 3-5 minutes for the rest of you.

Oh - final points. You will see from the help that one option is to 'Bind' your branch to the parent branch. This is a check-box option when you create your local branch. There is also a menu item to Bind or Unbind a branch. You might think this is useful. Basically if the branch is Unbound, when you commit a change it goes as far as your branch. To get it into the server you have to Push your revisions 'upstairs'. In contrast, if your branch is bound to the parent (the branch on launchpad) then when you commit changes they are sent BOTH to your local repository AND to the launchpad server - and the key thing is that if the commit to launchpad fails, the commit to your PC will fail too. So which to choose? An unbound branch can be worked on entirely offline whereas a bound branch needs an on-line connection to do a commit. But a bound branch cannot get out of synch. I think I would say that since we have worked happily all this time with checkouts which need an on-line connection to do the smallest task, we should use bound branches. Then your commits go to both locations and you never need to 'Push'.

Once you have read this and tried a branch for yourself, here are the help sections that I would recommend you to look at (Note how the locations all tend to be folders on the local PC).....
doc.bazaar.canonical.com/explore ... itory-view
doc.bazaar.canonical.com/explore ... tml#branch
doc.bazaar.canonical.com/explore ... tml#commit
doc.bazaar.canonical.com/explore ... -revisions
doc.bazaar.canonical.com/explore ... ind-branch

Oh another nice thing - the TortoiseBazaar integration in Windows Explorer is now instant too, which is very nice. I think my internet router will not need resetting anywhere near as much (if at all).

If this all works for you the same as it has for me, we should discuss whether to go back to the latest stable version of Bazaar. And one day we should discuss whether to throw away a lot of the early history of the project. 260,000 revisions is a lot of overhead to carry around when we haven't really released much to the wild. I haven't investigated whether Bazaar has a command to 'delete' revision history prior to a certain item, but I guess it does. I doubt that you have to start a new repository and add the current files to it.

Have fun and be amazed!!!!

alanjfpaterson

A few more thoughts overnight.....

Once you have a branch on your PC, which is a complete repository, you can create branches inside that - see the help docs. So when we come to do our own work on a limited issue bug it makes sense to create a local branch inside your repository. So I would, for example, create dev_0001505_daily_exch_rate as a branch within my personal repository. I would make my commits in that branch as usual and when I was done I would push my changes into my trunk. Then when I looked at my trunk it would have changes which I could commit which would send them up to the server. When I looked at the history log - or any of you looked at the history log - my check-ins would still show up as a branch off my repository rather than off the trunk one - but that is fine. The whole history would ultimately be known to global trunk once the commits reached it.

This is a much better way of working. It means there would be fewer server branches and more single-issue personal ones, which has got to be the way that Canonical develop Ubuntu! Individual developers would only create a new branch from the server rarely (so my 24 minutes to do it is hardly an issue).

One suggestion (I am sure there are plenty of others) is that our global trunk effectively contains the 'most recent stable release'. Then when we start on a new 'version' we create a server branch for it which we all create our own branch from. We all need read/write access to this server branch - but because we all derive from it, it becomes our current working code source and we would all be effectively checking each other's work during the development of this release. (Effectively that happens now but we are all using trunk as our ultimate source so we do not have the concept of stable code and development code that people can get at - in contrast to other open source projects.) When we approach a release date we would make sure that all our personal branches that we were happy with make it into the server and on release day we commit everything from our 'development version' server branch into trunk - and it becomes our stable release.

If you do something like that then for individual developers it would work like this:

Having got your local working branch from the server branch, you make a branch from your own trunk for your development. Keep committing to that.
When you want to get other people's changes to your PC, you 'Pull' them into your trunk (because now we are working collaboratively our branches are all children of the same server branch) - don't Merge
When you want to combine your work with everyone else's, Push your branch changes into your trunk. This is where you might have to resolve conflicts.
Commit these pending resulting changes to your trunk. If your branch is bound this will also push the changes to the server where everyone else can Pull them

So you can see that there is now not really any merging. We are working collaboratively and we do everything with Pull/Commit/Push. As now, only a few people need write access to trunk because trunk only gets updated when we do critical releases.

The final geeky point about this suggestion concerns what happens to work in progress when we 'close' our working server branch for a release. It would be quite likely that there would be some personal branches that we were not ready to commit. That's ok. I would make sure my branch is unbound. I would commit my changes (as far as they were complete) to my (now defunct) 'old' branch. Now I would create my new branch for the next version from the server. This is where the team would be committing all future work for the following release. Now I would create a new branch inside this repository (just like I had in the previous repository) and MERGE the changes from my old dev branch to the new dev branch. Then I would carry on working in my new repository. So this becomes the only time I would now be using the Merge command.

So what use is a Checkout?? The help gives a few clues. Checkouts just get a copy of the files without any branch information. So one possible use is that you want to copy the files from your branch to a USB Hard Drive that you want to take somewhere else. You edit the files and bring them back so that you can commit these changes back to your branch. Or you have two computers on a home network and you want to be able to work on either machine. Or possibly that you have a really wacky idea that may or may not work, so you checkout the files to a different folder location. You make lots of changes. If it works you can commit back to your branch - if it doesn't you haven't messed with any of the files in the branch folder. But basically the Checkout capability is not something you would normally use.

Final thought.... One of the nice things about every developer having the complete repository history is that we all effectively backup what is on the server. At the moment I don't think we have a backup anywhere. We only have the history of OpenPetra as long as we have a working connection to the launchpad server. Otherwise all we know is the most recent version.

pokorra

I remember that MatthisG wanted to set up Bazaar as a centralized VCS, so that changes are reproducable??? I cannot remember the details.
He liked that a local commit would be reflected on the server as well.

What about backups on the local machine? If we develop on a local branch, that is not backed up on the server, that is a greater risk than Launchpad losing data.

alanjfpaterson

I'm not sure what you mean by reproducible?

Regarding the commit to the server - that is exactly the point of a bound branch. If my branch is bound to the server (which I recommended in my original post) then my commit to my branch also commits to the server - and the whole commit fails if that does not actually happen. So in effect that works exactly as we do now - when a developer commits to his branch it also commits to the server. So at that time the backup exists on the server and on my PC - and then when other developers pull my change we all have got a backup. So actually the system is better than we have now.