This is to follow up from the discussion yesterday during the UEFI hacking sprint at the Linaro offices.

I've been playing around with git and repo to figure out how best to manage Linaro's UEFI tree. First, a summary of the methods I've looked at:

1) git sub-modules 2) git sub-trees 3) repo

1) git sub-modules A submodule is and external reference from one git repo to another so that the submodule appears as a sub directory in the working directory. The link is by reference using the SHA1 id from the submodule's git tree and none of the submodules files are tracked in the master tree.

Git keeps track of both the submodule's git url and SHA1 id as files in revision control so that the specific version of the submodule gets tracked. Changes to the submodule upstream are pulled in by committing a change to the SHA1 id (the tool takes care of doing this correctly)..

Changes to a submodule must be pushed out to the submodule git URL before the master tree will be able to see them. Care must be taken to not push out submodule references that aren't already in the submodule's master repository.

Work flow requires end users to do extra steps to fetch submodules after cloning a tree:

References appear to be only by SHA1 id, not by tag or branch. It's difficult to see at a glance what branch a sub module comes from. However, going into the subtree and doing a "git pull" does appear to work for tracking the submodule's git repo.

Sub module must be entirely contained in a single subdirectory path. No complex fan out across the repo allowed

2) git subtrees Git subtrees pull a copy of another git tree directly into the git repository. May optionally include the history of the subtree.

Files become part of the master repo and so one repo contains everything needed by users. If history is include, then all of the subtrees commits become part of the git history of the whole tree.

Same as with submodules, a subtree is attached to a specific point in git-history time. The commit text could state which branch was merged from the subtree repo, but just looking at a subtree merge commit doesn't say what branch it came from, or which branch should be tracked for updates

Subtree can either be constrained to a single subdirectory, or make changes across the tree. If the subtree directory layout exactly matches the master repo (at the same directory level, not in a subdirectory) then the git history becomes transparent. When in a subdirectory, git log won't follow the file history across the subtree merge point.

For end users git subtrees are transparent. Clone the tree and go.

3) Repo External tool on top of git Fetches and extracts multiple git repos into a single complex of working directories https://gerrit.googlesource.com/git-repo - Integrated with 'Gerrit" review tool, but does not require using gerrit - Each repo gets a seperate directory. It doesn't look like one can be a subdirectory of another, but it should be possible to create symlinks between them - The repo can be configured to follow a branch on each tree, or be locked to a specific tag. All is stored in a git repo so is kept under revision control. - Figuring out how to write a repo default.xml manifest file wasn't obvious, but it is a one-time setup thing. Once working it won't need to be manipulated frequently. - A canned repo file could make it easy to pull in the various sub projects needed to build a working UEFI image. - Possible downside; repo *really* likes to update itself from the upstream repo .git tree. That is potentially a risk if the upstream repo moves or otherwise stops working. No data would be lost, but it could be a denial-of-service kind of error.

Okay, that ended up being a long email. I've got some examples below, but I'll save a discussion of how best to maintain the linaro uefi rollup tree to a separate email.

g.

Sub-module Examples: Creating a submodule: $ git submodule add git://path/to/submodule/git SecretSubModule $ git status # On branch master # Your branch is ahead of 'origin/master' by 39 commits. # # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: .gitmodules # new file: SecretSubmodulePkg # $ git diff --staged diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..95e9d58 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "SecretSubmodulePkg"] + path = SecretSubmodulePkg + url = git:///path/to/submodule/git diff --git a/SecretSubmodulePkg b/SecretSubmodulePkg new file mode 160000 index 0000000..b4e3b57 --- /dev/null +++ b/SecretSubmodulePkg @@ -0,0 +1 @@ +Subproject commit b4e3b579a47bc67011738fa938bbbc3758826f4c $ git comiit

Updating submodule: $ cd SecretSubmodulePkg/ $ git pull remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. remote: Total 3 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (3/3), done.

From file:///home/grant/hacking/uefi-working/SecretPkg

b4e3b57..a9859ea master -> origin/master Updating b4e3b57..a9859ea Fast-forward README | 2 ++ 1 file changed, 2 insertions(+) $ cd .. $ git diff diff --git a/SecretSubmodulePkg b/SecretSubmodulePkg index b4e3b57..a9859ea 160000 --- a/SecretSubmodulePkg +++ b/SecretSubmodulePkg @@ -1 +1 @@ -Subproject commit b4e3b579a47bc67011738fa938bbbc3758826f4c +Subproject commit a9859ea6ebbf196a5398676d342c3257fb00dd3f $ git commit -a [master bb730a6] Pull in latest SecretSubmodulePkg submodule 1 file changed, 1 insertion(+), 1 deletion(-)

Retrieving a git tree with submodules (both master and submodule trees need to be accessable to user): $ git clone git://path/to/master/git master $ cd master $ git submodule init $ git submodule update

Repo example: First create a new git repo to contain the manifest file: $ mkdir uefi-manifest $ cd uefi-manifest $ git init-db $ cat > default.xml << EOF <?xml version="1.0" encoding="UTF-8"?> <manifest> <remote name="origin" fetch="git://path/to/git/repos/"/> <default remote="origin" revision="master"/> <project name="edk2-sourceforge" path="EDK2"/> <project name="Origen-EDK-II-Package" path="Origen"/> <project name="SecretPkg" path="SecretPkg"/> </manifest> EOF $ git add default.xml $ git commit $ cd ..

Download repo $ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo $ chmod a+x ~/bin/repo

Then initialize the repo: $ mkdir working-tree $ cd working-tree $ repo init -u git://path/to/manifest/git/repo.git $ repo sync

Now all the git trees will be fetched and copied into the working directory. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd.