SVN update on live sites to r43514, questions on branching

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

SVN update on live sites to r43514, questions on branching

Brion Vibber-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've got the live Wikimedia site all up to date at r43514 now.

We'd been updating some extensions individually over the last couple
weeks, but full updates were held back as general code review got a
little behind during the lead-up to the fundraiser and our staff meeting
last week... I'm hoping to get us on a regular weekly update schedule,
probably Tuesdays since my experience is that Mondays end up totally
unproductive. :)

I did have to pull back the Special:Search redesign for the moment; it's
looking *awesome* but has a few glitches still, which I'm hoping we can
resolve before putting it live.

See my comments on issues I noticed in the revert:
http://www.mediawiki.org/wiki/Special:Code/MediaWiki/43499


I'm thinking we should start making more active use of branches for
experimental/iterative development like this, where existing features in
core are majorly refactored and need some iterations of testing before
going live.

We try to keep our trunk code ready-to-run at all times, so when
something in trunk is not quite ready yet we end up rolling it back
(which requires tracking down multiple changes and reverting all of
them) or else rushing fixes so we can get an update pushed out.


The SVN server was updated to 1.5 a while ago, which is apparently a
little handier at branch merging, but branching still is kind of awkward
in SVN. Any good recommendations on SVN-friendly DVCSs? I know some
folks use SVK or a GIT-SVN bridge for doing various local development,
but how easy is it to share a development branch among multiple
developers over time this way?

- -- brion
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkkeIYgACgkQwRnhpk1wk44rIQCeP53GOXDdKNxpPcMkQDM9Getz
9PUAniho/3lOK7r+omsCyxEGn+9PPVdt
=x3IC
-----END PGP SIGNATURE-----

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Bugzilla from andrew@epstone.net
On Sat, Nov 15, 2008 at 12:10 PM, Brion Vibber <[hidden email]> wrote:
> The SVN server was updated to 1.5 a while ago, which is apparently a
> little handier at branch merging, but branching still is kind of awkward
> in SVN. Any good recommendations on SVN-friendly DVCSs? I know some
> folks use SVK or a GIT-SVN bridge for doing various local development,
> but how easy is it to share a development branch among multiple
> developers over time this way?

I'm a big fan of git, having set it up recently (also ask Aryeh/Simetrical).

Branching and tagging works nicely and natively - git encourages,
rather than tolerates the non-linear development model, and has some
nice tools to keep branches in-sync with trunk (unlike svn, which
requires devil invocations and black magic).

In terms of architecture, we can have a central server which holds
trunk, all of the development branches, and so on. Development
branches can be pushed to the central server, and pulled by people
wanting to use them. Switching between branches is as simple as 'git
checkout branch-name'. We can, of course, have separate central
servers if we wish (since that's kinda the point of a DVCS).

--
Andrew Garrett

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Juliano F. Ravasi
In reply to this post by Brion Vibber-3
Hello Brion,

Brion Vibber wrote:
> The SVN server was updated to 1.5 a while ago, which is apparently a
> little handier at branch merging, but branching still is kind of awkward
> in SVN.

That is probably not branching that you meant, but merge tracking.

There have been reasonable merge tracking support available for a long
time for Subversion, either through the svnmerge.{sh,py} utility (that
is distributed along with Subversion itself, since 1.2) or through SVK
(both existed since before Git)... and *nobody cared*! People either
branched and merged anyways, like described in svnbook (until 1.5), or a
few looked for these handier tools when they needed.

Then, Torvalds did that presentation, and for some reason now everyone
is scared of branching in Subversion... go figure.

Subversion 1.5 incorporates good branching support (not great, or
excellent... just good enough). This kind of branch you suggested is
quite trivial to handle now in Svn 1.5. It is just a matter of giving
another read to chapter 4 of svnbook:

http://svnbook.red-bean.com/en/1.5/svn.branchmerge.html

> Any good recommendations on SVN-friendly DVCSs?

The most Subversion-friendly DVCS is SVK. It exists since before
Subversion 1.0, and have been continuously evolving along with
Subversion since then. It supports the same features that are unique to
Subversion, like directory versioning and properties, and uses the same
underlying repository, so, it plays really well with Subversion. Your
local revisions are a perfect mirror of remote revisions (with the
exception of revision numbers).

There is Git, which uses a different and conflicting model from
Subversion. Git doesn't care about directories, properties and copy/move
information, and this makes it easy for people to silently screw up your
repository (properties that should be updated aren't, file references
that should be placed aren't). Some of these problems can be partially
avoided by threatening your co-developers if they don't use
--find-copies-harder to git svn dcommit, others are simply disregarded
because Git ignores it completely.

Bazaar has some so-so Subversion support, but also has other problems.
Bazaar doesn't have the notion of a file copy, and also doesn't support
all features that are unique to Subversion, so conversion is also lossy.

Mercurial has a few ways to communicate with Subversion, all of them
still in development.

> I know some
> folks use SVK or a GIT-SVN bridge for doing various local development,
> but how easy is it to share a development branch among multiple
> developers over time this way?

Git has a lot of mind-share, most from people who never cared about
distributed development, then watched Torvalds presentation and now
think that Git is the best tool for everything, just because "Linus said
so". The fact that GNU arch, Monotone, Darcs and SVK were sitting around
way before Git was planned is just a minor detail, most of these people
will refuse to even consider testing the alternatives to check what they
have to offer.

In this case, SVK has a clear advantage over Git, because you don't plan
to convert the main repository away from Subversion yet.

In your use-case, you want public branches and private branches. The
git-svn bridge is more intended to be used for a single developer
dealing with a public Subversion repository, or with some extra work, a
Git-based project to be exported to a read-only Subversion repository.
It is not trivial to share Git branches among developers and still be
able to use git-svn to commit any of these shared branches back to the
central repository. Expect things to break.

In the case of the Special:Search redesign, it sounds like a public
branch, so it makes sense to have this branch in the central repository.
Then SVK can be used, since it understands Subversion branches and its
own local branches. SVK merge tracking is still incompatible with
Subversion 1.5 merge tracking, so it is a good idea to stick to a single
tool for branch <-> trunk merges (private branch <-> public branch
merges and other work are fine with either tool).

Also SVK mirroring is another strong point. It is way better than the
scavenging that git-svn does through svn logs to find what and how to
convert the repository. Do you remember that I asked you on IRC about
providing an svn dump of the MediaWiki repository? SVK is able to use
such a dump to bootstrap the mirroring process, making the initial
repository clone *a lot* faster than git-svn.

This is all only while you want to keep the Subversion central repo. If
you really want to go distributed, SVK would be inviable, and Git is
probably not the best option.

Best regards,
Juliano.

--
Juliano F. Ravasi ·· http://juliano.info/
5105 46CC B2B7 F0CD 5F47 E740 72CA 54F4 DF37 9E96

"A candle loses nothing by lighting another candle." -- Erin Majors

* NOTE: Don't try to reach me through this address, use "contact@" instead.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Siebrand Mazeland
Hi Juliano,

Thank you for that elaborate analysis. Aside from functionality comparison,
I find one thing missing that would be important if the MediaWiki project
were to facilitate another tool for SCM: multi platform support. Can you add
some information on that, too, because I know that we have developers on
Linux, OS X, and Windows, and we wouldn't really like to exculde one of
those groups just because of the used SCM tool. Having proper clients on all
major platforms is one of the reasons I really like Subversion for...

Cheers! Siebrand

-----Oorspronkelijk bericht-----
Van: [hidden email]
[mailto:[hidden email]] Namens Juliano F. Ravasi
Verzonden: zaterdag 15 november 2008 4:32
Aan: Wikimedia developers
Onderwerp: Re: [Wikitech-l] SVN update on live sites to r43514, questions on
branching

Hello Brion,

Brion Vibber wrote:
> The SVN server was updated to 1.5 a while ago, which is apparently a
> little handier at branch merging, but branching still is kind of
> awkward in SVN.

That is probably not branching that you meant, but merge tracking.

There have been reasonable merge tracking support available for a long time
for Subversion, either through the svnmerge.{sh,py} utility (that is
distributed along with Subversion itself, since 1.2) or through SVK (both
existed since before Git)... and *nobody cared*! People either branched and
merged anyways, like described in svnbook (until 1.5), or a few looked for
these handier tools when they needed.

[..]


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Platonides
In reply to this post by Juliano F. Ravasi
Juliano F. Ravasi wrote:
> Do you remember that I asked you on IRC about
> providing an svn dump of the MediaWiki repository? SVK is able to use
> such a dump to bootstrap the mirroring process, making the initial
> repository clone *a lot* faster than git-svn.

You could use svnsync to get a copy of the repository.
Although I don't know about the speed of both approaches.


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Chad
In reply to this post by Siebrand Mazeland


Fwiw, git works on windows now, at least to
some degree. I know there's a plugin for
Aptana now and I'm assuming it will work for
Eclipse in general.

-Chad

On Nov 15, 2008 5:07 AM, "Siebrand Mazeland" <[hidden email]> wrote:

Hi Juliano,

Thank you for that elaborate analysis. Aside from functionality comparison,
I find one thing missing that would be important if the MediaWiki project
were to facilitate another tool for SCM: multi platform support. Can you add
some information on that, too, because I know that we have developers on
Linux, OS X, and Windows, and we wouldn't really like to exculde one of
those groups just because of the used SCM tool. Having proper clients on all
major platforms is one of the reasons I really like Subversion for...

Cheers! Siebrand

-----Oorspronkelijk bericht-----
Van: [hidden email]
[mailto:[hidden email]] Namens Juliano F. Ravasi
Verzonden: zaterdag 15 november 2008 4:32
Aan: Wikimedia developers
Onderwerp: Re: [Wikitech-l] SVN update on live sites to r43514, questions on
branching

Hello Brion, Brion Vibber wrote: > The SVN server was updated to 1.5 a while
ago, which is apparen...
[..]

_______________________________________________ Wikitech-l mailing list
[hidden email]...

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Aryeh Gregor
In reply to this post by Juliano F. Ravasi
On Fri, Nov 14, 2008 at 10:31 PM, Juliano F. Ravasi <[hidden email]> wrote:
> That is probably not branching that you meant, but merge tracking.

If you're going to correct someone's use of technical terminology like
that, maybe you should explain what you think the difference is?  To
me, "branching" including the entire process of creating, maintaining,
and re-merging branches, which seems to be precisely what Brion means.
 Subversion doesn't do that as nicely as DVCSes, as far as I know.

> Then, Torvalds did that presentation, and for some reason now everyone
> is scared of branching in Subversion... go figure.

Because before then nobody knew what they were missing.  :)

> Subversion 1.5 incorporates good branching support (not great, or
> excellent... just good enough). This kind of branch you suggested is
> quite trivial to handle now in Svn 1.5. It is just a matter of giving
> another read to chapter 4 of svnbook:
>
> http://svnbook.red-bean.com/en/1.5/svn.branchmerge.html

From reviewing that, it seems to still handle merging as a diff/patch
operation.  It doesn't copy over the commits, it only copies over the
changes.  Therefore you inevitably get giant merge commits that
clutter up commit histories and/or are impossible to actually review.
Correct?  This is the really nice thing about DVCS' branch handling to
me.

> The most Subversion-friendly DVCS is SVK. It exists since before
> Subversion 1.0, and have been continuously evolving along with
> Subversion since then. It supports the same features that are unique to
> Subversion, like directory versioning and properties, and uses the same
> underlying repository, so, it plays really well with Subversion. Your
> local revisions are a perfect mirror of remote revisions (with the
> exception of revision numbers).
>
> There is Git, which uses a different and conflicting model from
> Subversion. Git doesn't care about directories, properties and copy/move
> information, and this makes it easy for people to silently screw up your
> repository (properties that should be updated aren't, file references
> that should be placed aren't). Some of these problems can be partially
> avoided by threatening your co-developers if they don't use
> --find-copies-harder to git svn dcommit, others are simply disregarded
> because Git ignores it completely.

FWIW, here's a very detailed account of how to use git svn and
arguments on why it's better than SVN alone or SVK, from someone who
switched from SVK to git: "I used to push strongly for SVK, but got
brow-beaten by people who were getting far more out of their version
control system than I knew possible until I saw what they were talking
about." http://utsl.gen.nz/talks/git-svn/intro.html

git not caring about directories is not a big deal (we have rather few
empty directories in trunk, and most of those could readily be dropped
or have an empty index.html or something dumped in them).  Copy/move
info might be a pain, although of course that would be solved by
having more people use git instead of SVN.  :D

> Git has a lot of mind-share, most from people who never cared about
> distributed development, then watched Torvalds presentation and now
> think that Git is the best tool for everything, just because "Linus said
> so". The fact that GNU arch, Monotone, Darcs and SVK were sitting around
> way before Git was planned is just a minor detail, most of these people
> will refuse to even consider testing the alternatives to check what they
> have to offer.

You evidently dislike git a lot.  Others (like Andrew, me, the
developers of the Linux kernel/Wine/...) happen to like it quite a
bit, and we seem to be in the majority of people who have tried it,
anecdotally.  I have, by the way, tried Mercurial, and didn't like it
as much.  That it took public backing by a major figure in the
open-source community to get people to see the advantages in DVCSes
doesn't need to be held against them.

> In this case, SVK has a clear advantage over Git, because you don't plan
> to convert the main repository away from Subversion yet.

On the other hand, it has the clear disadvantage of being less widely
used and supported, I suspect, by this point (maybe I'm wrong there?).
 And certainly I don't know of any current developers who use it,
unlike git -- which is used by two MediaWiki developers now.  (Maybe
I'm wrong there too?)  But I've never used it, and so can't say much
about it.

There are some things I like a lot about git that SVK evidently
doesn't support, like easily rewriting history for private commits.
Other notable criticisms gleaned from the page I mentioned above (SVK
user who switched to git):

"SVK claims on its home page to be distributed, but by everyone else's
definition, it's not, because it's not decentralised - there's always
an upstream. No, SVK merely offers disconnected operation. If I meet
you in the middle of a cruise and we both have a mirror of a
subversion repository, I just can't easily, natively share my local
branch with you if we're both on SVK. "

"git normally stores its repository information under .git at the top
level of your checkout. But everything's compressed and the filenames
don't resemble the files in your checkout so grep -r and find etc
don't hate you."

"I don't know about you but I was always running into situations where
my ~/.svk/config didn't match reality, and there were no breadcrumbs
left in the checkout to do anything with it. I much prefer these
floating repositories and there was some talk of adding them to SVK. "

> In your use-case, you want public branches and private branches. The
> git-svn bridge is more intended to be used for a single developer
> dealing with a public Subversion repository, or with some extra work, a
> Git-based project to be exported to a read-only Subversion repository.
> It is not trivial to share Git branches among developers and still be
> able to use git-svn to commit any of these shared branches back to the
> central repository. Expect things to break.

I don't know how much benefit you'd get from tracking these branches
on git anyway, when the whole rest of the repo is still on SVN.  It
would be somewhat nicer to maintain, maybe, but merging it back in
would still require a giant merge commit.

I haven't done any branching in git, to be honest (only rebasing for
private patch sets), so I can't give an informed opinion on how best
to go about this even if we stuck purely to git instead of throwing in
git/SVN interactions to confuse things even further.

On Sat, Nov 15, 2008 at 5:05 AM, Siebrand Mazeland <[hidden email]> wrote:
> Thank you for that elaborate analysis. Aside from functionality comparison,
> I find one thing missing that would be important if the MediaWiki project
> were to facilitate another tool for SCM: multi platform support. Can you add
> some information on that, too, because I know that we have developers on
> Linux, OS X, and Windows, and we wouldn't really like to exculde one of
> those groups just because of the used SCM tool. Having proper clients on all
> major platforms is one of the reasons I really like Subversion for...

git works on all Unix-likes (including Linux and OS X) fine by command
line.  It will also likely work in Cygwin without much trouble (again,
on the command line).  There are Windows GUIs available for git as
well, which are apparently somewhat usable, but they didn't seem as
polished as Mercurial's last I checked.  I don't know about the others
being discussed.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Marco Schuster-2
On Sun, Nov 16, 2008 at 12:38 AM, Aryeh Gregor
<[hidden email]> wrote:
> git works on all Unix-likes (including Linux and OS X) fine by command
> line.  It will also likely work in Cygwin without much trouble (again,
> on the command line).  There are Windows GUIs available for git as
> well, which are apparently somewhat usable, but they didn't seem as
> polished as Mercurial's last I checked.  I don't know about the others
> being discussed.
Git AFAIR even has a native commandline client on Windows.
Which has some problems:
a) speed, it is slower than on *nix
b) the commandconsole itself, cmd.exe is bitrotten from down to top,
look at command history for excellent examples. not everyone has the
SFU installed.
c) AFAIR(!) it doesn't support all commands of *nix git

Cygwin of course is the other possibility, but here applies the
problem of speed and the work amount needed to install it. It can
really suck to get a working cygwin environment...

Marco

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Juliano F. Ravasi
In reply to this post by Siebrand Mazeland
Hello Siebrand,

Siebrand Mazeland wrote:
> Hi Juliano,
>
> Thank you for that elaborate analysis. Aside from functionality comparison,
> I find one thing missing that would be important if the MediaWiki project
> were to facilitate another tool for SCM: multi platform support. Can you add
> some information on that, too, because I know that we have developers on
> Linux, OS X, and Windows, and we wouldn't really like to exculde one of
> those groups just because of the used SCM tool. Having proper clients on all
> major platforms is one of the reasons I really like Subversion for...

Yeah, that is something that Subversion still shines bright compared to
all other tools.

If you don't mind using the command line in Windows, Git, Mercurial and
Bazaar all have good command line support, virtually the same between
different OSes. But usually, Windows users don't like the command line,
and here is where it makes a difference.

Git is only officially supported in Windows in Cygwin, which makes it
slow as a rock, and definitely no real Windows integration. Git itself
was not designed to run on non-Posix platforms.

Then there is the msysGit fork that strives to "fix" that, and have Git
compile on MingW32. The problem, again with Git original design, is that
msysGit end up depending on bash, perl, tcl, tk, vim and other GNU ports
from MingW32, most of these are aliens in a pure-Windows environment.
You mostly have to run msysgit in its own private bash shell, which
redefines the meaning of some Windows commands, like find, in order to
match those in Linux (that is expected from Git's porcelain). Git-svn is
another pain. MsysGit also offers virtually no Windows integration. It
comes with some standalone interface (git-gui, gitk), and there are
plans of a Tortoise-like shell integration called git-cheetah, not ready
for consumption.

People argue that Git works "fine" in Windows through msysGit, but
usually for people with good Linux experience, that fell comfortable in
a bash shell. Most Windows developers don't match this profile.

Mercurial provides *much* better Windows support, since it is designed
to run over Python with whatever it provides and making few assumptions
about the underlying operating system. Python is natively supported on
Windows, so the end result is much better than Git. It had a few
problems in the past (e.g. with filesystem case-insensitivity), but that
is fixed by now. Two-way Svn communication is still under development.

There is TortoiseHg, that provides Windows integration, and supports
most operations the user usually needs, but it's interface is currently
really ugly, in Gtk+ (not Windows GDI native), with buttons in strange
places, etc. It shares libraries with TortoiseSVN 1.5, and works really
well. It includes kdiff3, which is a great tool for merging.

Bazaar shares the same advantages of being written in Python with
Mercurial. Bazaar properly handles Unicode filenames in different
locales (something that both Git and Mercurial fail), and is the most
user-friendly of the three. TortoiseBzr is included with the default
Windows installer, has better i18n support (more friendly for
non-English speakers), and is really well polished compared to
TortoiseHg; but it is still experimental, and a lot of functionality is
either missing or broken. It has somewhat good two-way Svn communication.

And, in case you want to keep your central Subversion repository, there
is SVK. It works great in Windows, but is just a command-line tool.
There is no GUI, but you have the advantage that all your current
Subversion tools (specially, TortoiseSVN and Eclipse) are completely
usable in a distributed manner, without modifications. In this case you
use the SVK command-line to mirror, pull, push and merge, and Subversion
for the local branch development. Not really user-friendly, but works
well most of the time.

I have some big interest (as in scientific research interest) in version
control systems. I give presentations and short courses about VCSes to
teams of developers, most of them PHP, Java and C# developers, most of
them in Windows environment.

The first thing that I notice clearly is that Windows developers, who
don't have a lot of contact with Linux or open-source, don't care a lot
about distributed version control. For them, having a user-friendly,
feature-complete, fully-integrated UI like TortoiseSVN and Subeclipse is
way more important than being capable of sharing commits in a
distributed manner. They recognize the benefits of DVCSes, but they are
more objective: "Do I really need these specific features? Will them
give me a raise? No? So thanks."

Those who are more enthusiastic and want to jump into distributed
version control, they find Mercurial and Bazaar much easier (and saner)
to use than Git. I was once told that there is no point in spending too
much time dealing and learning about a single version control tool, even
though it offers more features (referring to Git over Hg), than doing
actual work.

That sounded sensible, and made me think about the fact that most DVCS
fuss came after Torvalds presentation (even though they weren't any news
for quite a time already), and mostly around Git, and mostly moved by
open-source enthusiasts. It really looks like the "Linus said so"
syndrome. Open-source enthusiasts tend to regard Torvalds as a deity,
and give him too much credibility, even when his mouth is just expelling
a lot of bullsh*t.

--
Juliano F. Ravasi ·· http://juliano.info/
5105 46CC B2B7 F0CD 5F47 E740 72CA 54F4 DF37 9E96

"A candle loses nothing by lighting another candle." -- Erin Majors

* NOTE: Don't try to reach me through this address, use "contact@" instead.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Juliano F. Ravasi
In reply to this post by Platonides
Platonides wrote:
> You could use svnsync to get a copy of the repository.
> Although I don't know about the speed of both approaches.

What SVK does is very similar to svnsync, and that is the point. The
svnsync is *slow* over the network. With a dumpfile, SVK allows a very
fast initial mirroring, using the svnsync protocol only for the later syncs.

--
Juliano F. Ravasi ·· http://juliano.info/
5105 46CC B2B7 F0CD 5F47 E740 72CA 54F4 DF37 9E96

"A candle loses nothing by lighting another candle." -- Erin Majors

* NOTE: Don't try to reach me through this address, use "contact@" instead.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Juliano F. Ravasi
In reply to this post by Aryeh Gregor
Aryeh Gregor wrote:
> If you're going to correct someone's use of technical terminology like
> that, maybe you should explain what you think the difference is?  To
> me, "branching" including the entire process of creating, maintaining,
> and re-merging branches, which seems to be precisely what Brion means.

Branching is a more general term. An svn checkout, for example, is a
branch. Not the kind of named branch you create in the repository, but a
small, one-level branch, nevertheless.

Most branches created in Subversion are release branches and tags
(create-only, locked branches), that either don't require merge
tracking, or share only single commits between them.

Also, Subversion supports the idea of inner branches, that no other
version control (AFAIK) supports. Like:

   cd skins/
   svn cp Monobook.php MyOwnSkin.php
   svn ci -m "branched Monobook skin"
     /* make changes in MyOwnSkin.php */
   svn ci -m "changes to my skin"
     /* time passes */
   svn update
     /* changes to Monobook are received */
   svn merge $repo/skins/Monobook.php MyOwnSkin.php
   svn ci -m "merged changes from Monobook to MyOwnSkin"

> From reviewing that, it seems to still handle merging as a diff/patch
> operation.  It doesn't copy over the commits, it only copies over the
> changes.  Therefore you inevitably get giant merge commits that
> clutter up commit histories and/or are impossible to actually review.
> Correct?

Not really. To explain, I'd have to dig too much into the inner workings
of both Git and Svn. But in short, they both boil down to very similar
procedures. The difference is in how this information about the merge is
stored and interpreted. Also, since Svn was born as centralized, the
information registered and immediately available on svn log doesn't
match what a Git user would expect. It is possible (but still
cumbersome) to check the diffs in svn:mergeinfo properties, and extract
the same information very similar to what Git stores.

Also, every commit in Git has a unique SHA1 identifier, that is
perceived in different branches. In Svn, every commit is referenced by
where it occurred first (in the branch where it happens with the lowest
revision ID), then this information (branch path + revision id) is
copied into other branches svn:mergeinfo properties to indicate that
that commit was transplanted.

Note also that Svn merge tracking is more complex (in an algorithmic
point-of-view) than Git's and Mercurial's, since Svn supports detachable
directories and partial checkouts. This same extra complexity is what
enables the support for merge tracking of inner branchs I mentioned above.

> FWIW, here's a very detailed account of how to use git svn and
> arguments on why it's better than SVN alone or SVK, from someone who
> switched from SVK to git: "I used to push strongly for SVK, but got
> brow-beaten by people who were getting far more out of their version
> control system than I knew possible until I saw what they were talking
> about." http://utsl.gen.nz/talks/git-svn/intro.html

I'll take a look at it later. But I agree that using other DVCS with
less restrictions than SVK is better. But if you know how to use SVK
correctly, it plays nicer with the upstream Svn repository than the
git-svn bridge. That is from my own experience.

I'm suggesting SVK just because AFAIK MediaWiki is not yet planning to
move to distributed version control. When that day comes, then surely
there are much better tools than SVK.

> git not caring about directories is not a big deal (we have rather few
> empty directories in trunk, and most of those could readily be dropped
> or have an empty index.html or something dumped in them).

Directory version isn't just about empty directories. Directory
versioning also plays a big role for move/rename history tracking, merge
tracking and version control metadata (Svn properties).

Simple example: you rename a directory with a thousand versioned files
inside. You don't make any changes to the files, just the directory name.

In Svn, it is recorded as a single directory move operation, all files
register *no* change, since they actually weren't touched.

In Git, it is recorded as a thousand files removed, plus a thousand
files added, which is a *wrong* description of what happened. Also,
since Git doesn't record rename history upon commit (that happens only
once), this information has to be figured out later (that may happen
many times) using an exponential-complexity algorithm. It doesn't make
any sense when you think about it.

> Copy/move
> info might be a pain, although of course that would be solved by
> having more people use git instead of SVN.  :D

But this problem of copy/move information being ignored/discarded have
already caused so much pain to Git that I still wonder why they insist
in calling that a feature. Demanding the user to pass -M, -C and
--find-copies-harder options to find and use the correct information,
causing an exponential-complexity algorithm to be performed many times
over the same information that could be simply stored in the commit
manifest. It sounds completely illogical.

> You evidently dislike git a lot.

Sure. For many reasons (technical and ethical reasons). Kind off-topic
in this thread and this list.

> On the other hand, it has the clear disadvantage of being less widely
> used and supported, I suspect, by this point (maybe I'm wrong there?).

No, you are right. SVK is almost always forgotten. That means smaller
community, less documentation, less support, slower development, etc.

> And certainly I don't know of any current developers who use it,

I use it, with MediaWiki's repository. Using it, I can sync between MW's
repo, my local mirror, and my own public MW repo using it, something
that would be extremely difficult with git-svn.

> There are some things I like a lot about git that SVK evidently
> doesn't support, like easily rewriting history for private commits.

Rewriting history is something that SVK doesn't support by design. To
rewrite history means to lose information, and that is considered wrong
(even if you think that you won't need the overwritten history anymore).

That doesn't mean you can't *recreate* a new history as you like. You
rebranch from the point you want to recreate, do whatever changes you
want until the head of the former branch, then delete the former branch.
You have the same effect, but information is never lost forever. If you
find that you screwed up something trying to recreate the history, the
old history is still there to aid you.

> Other notable criticisms gleaned from the page I mentioned above (SVK
> user who switched to git):
>
> "SVK claims on its home page to be distributed, but by everyone else's
> definition, it's not, because it's not decentralised - there's always
> an upstream. No, SVK merely offers disconnected operation. If I meet
> you in the middle of a cruise and we both have a mirror of a
> subversion repository, I just can't easily, natively share my local
> branch with you if we're both on SVK. "

Not exactly true. It is true that SVK is optimized to work with upstream
repositories, but it is not true that you are unable to share local
branches without the *main* upstream.

You can have *multiple upstream repositories* in SVK. That may blow your
mind if you think that git-svn is the best bridge between your local
repository and the upstream Subversion repo.

SVK doesn't share branches directly with other SVK, but nothing blocks
you from setting up a temporary repository between you and your friend
that is on this cruise, replicate your local branch to this temporary
repository, and give him access to this temporary repository. It is just
a different way of developer-to-developer push that other DVCSes support.

It may be even simpler if you give him access directly to your internal
SVK depot, but you usually don't want him to have direct access to your
depot.

You have to know the tool you use. The problem is that very few people
know how to use SVK, very few people put the same effort they do into
learning Git, and take the easy path of talking bad about it.

> "git normally stores its repository information under .git at the top
> level of your checkout. But everything's compressed and the filenames
> don't resemble the files in your checkout so grep -r and find etc
> don't hate you."

SVK doesn't even create *any* bookkeeping directory, unless you order it
to do so. SVK checkouts are *completely* clean by default. So this is
even less of an issue for SVK.

> "I don't know about you but I was always running into situations where
> my ~/.svk/config didn't match reality, and there were no breadcrumbs
> left in the checkout to do anything with it. I much prefer these
> floating repositories and there was some talk of adding them to SVK. "

Because he didn't read the documentation. It was a design decision to
keep checkouts completely clean and have bookkeeping information stored
in a different private area. SVK provides commands to move checkouts,
and to relocate checkout paths if you move a working copy without using SVK.

And now (since 2.0 I think) SVK already supports floating working
copies, if you want.

Best regards,
Juliano.

--
Juliano F. Ravasi ·· http://juliano.info/
5105 46CC B2B7 F0CD 5F47 E740 72CA 54F4 DF37 9E96

"A candle loses nothing by lighting another candle." -- Erin Majors

* NOTE: Don't try to reach me through this address, use "contact@" instead.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Platonides
In reply to this post by Marco Schuster-2
> On Sun, Nov 16, 2008 at 12:38 AM, Aryeh Gregor (Simetrical) wrote:
>> git works on all Unix-likes (including Linux and OS X) fine by command
>> line.  It will also likely work in Cygwin without much trouble (again,
>> on the command line).

The problem of git on Windows is not executing it but executing it
/fast/. fork() implementations on Windows are painful and I think there
were also problems with Windows managing the filesystem in a less
efficient way than Linux.
For heavily FS based it can make a diffference.


> Marco Schuster wrote:
> b) the commandconsole itself, cmd.exe is bitrotten from down to top,
> look at command history for excellent examples. not everyone has the
> SFU installed.
What does Services For Unix provide to the command history that isn't on
vanilla cmd.exe ?



> Cygwin of course is the other possibility, but here applies the
> problem of speed and the work amount needed to install it. It can
> really suck to get a working cygwin environment...
>
> Marco

Cygwin is a bit intrusive on the system.


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: SVN update on live sites to r43514, questions on branching

Aryeh Gregor
In reply to this post by Juliano F. Ravasi
On Sat, Nov 15, 2008 at 9:09 PM, Juliano F. Ravasi <[hidden email]> wrote:
> Branching is a more general term. An svn checkout, for example, is a
> branch. Not the kind of named branch you create in the repository, but a
> small, one-level branch, nevertheless.

Well, okay.  You're probably using research terminology instead of
programming terminology.  By "branch" most of us mean what SVN, git,
etc. call a "branch".

> Most branches created in Subversion are release branches and tags
> (create-only, locked branches), that either don't require merge
> tracking, or share only single commits between them.

Okay, true.  I don't think that's what we're talking about.

> Not really. To explain, I'd have to dig too much into the inner workings
> of both Git and Svn. But in short, they both boil down to very similar
> procedures. The difference is in how this information about the merge is
> stored and interpreted. Also, since Svn was born as centralized, the
> information registered and immediately available on svn log doesn't
> match what a Git user would expect. It is possible (but still
> cumbersome) to check the diffs in svn:mergeinfo properties, and extract
> the same information very similar to what Git stores.

Okay, well, the implementation isn't of much concern to me.  I'm a
user, not a researcher.  I would like it to work nicely, that's all.

> I'm suggesting SVK just because AFAIK MediaWiki is not yet planning to
> move to distributed version control. When that day comes, then surely
> there are much better tools than SVK.

Well, okay.  The original request, though, was about branch merging
specifically.  Does SVK help with that at all?  If not, it's not
useful for the present purpose.

What is it, specifically, that people don't like about SVN branches,
anyway?  In my case, I don't like branches because I tend to create
them, switch back to the main repo to do something else, totally
forget about my branch, and then eventually delete it when it's
obvious I'm too lazy to ever finish my work on it.  :)  Or in other
words, I've never really *used* them.  And I've never used git
branches much either, only rebasing (which isn't good for publication,
although it's great for maintaining private patch sets).  So I can't
give much of an opinion here.

On the other hand, the way we often handle this kind of stuff is to
just let the two implementations sit in trunk together, divided by a
configuration setting, so devs can switch to the new one and the live
site still uses the old one, but everything is in trunk.  Then when
the new one is done, we remove the old one entirely.  (Or not.  Can
you hear me, Parser_OldPP?  Are you still alive?!)  This seems like a
pretty decent approach to the search issue.

> Directory version isn't just about empty directories. Directory
> versioning also plays a big role for move/rename history tracking, merge
> tracking and version control metadata (Svn properties).
>
> Simple example: you rename a directory with a thousand versioned files
> inside. You don't make any changes to the files, just the directory name.
>
> In Svn, it is recorded as a single directory move operation, all files
> register *no* change, since they actually weren't touched.
>
> In Git, it is recorded as a thousand files removed, plus a thousand
> files added, which is a *wrong* description of what happened. Also,
> since Git doesn't record rename history upon commit (that happens only
> once), this information has to be figured out later (that may happen
> many times) using an exponential-complexity algorithm. It doesn't make
> any sense when you think about it.

I don't really care, as long as it works, and so far it has for me --
without my having to remember to bother with "git cp" or similar
special commands.  As long as it's fast enough for what I'm doing,
implementational purity doesn't matter as much as convenience, and git
has a slight win in convenience here, at least at commit time.

> I use it, with MediaWiki's repository. Using it, I can sync between MW's
> repo, my local mirror, and my own public MW repo using it, something
> that would be extremely difficult with git-svn.

Well, Andrew has said he's using git-svn to sync his working copy
between multiple computers.  I don't know if he makes any of them
public.

> Rewriting history is something that SVK doesn't support by design. To
> rewrite history means to lose information, and that is considered wrong
> (even if you think that you won't need the overwritten history anymore).
>
> That doesn't mean you can't *recreate* a new history as you like. You
> rebranch from the point you want to recreate, do whatever changes you
> want until the head of the former branch, then delete the former branch.
> You have the same effect, but information is never lost forever. If you
> find that you screwed up something trying to recreate the history, the
> old history is still there to aid you.

Well, in git you can recover the history as well, if you try hard
enough and haven't run git-prune.  :)  Keeping the appropriate
original authorship dates when rewriting history is important when I'm
maintaining live patches to third-party software -- admittedly a
specialized case, but one of my main uses of git -- because I want to
know when the patch originally went live.  I gather your method
wouldn't do that.

> SVK doesn't even create *any* bookkeeping directory, unless you order it
> to do so. SVK checkouts are *completely* clean by default. So this is
> even less of an issue for SVK.

Although, it's a PITA for Subversion.  :(

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l