New instruction format. Got rid of ambiguous references, adopted @ notation. README updated accordingly

README.md

@@ -18,38 +18,19 @@ a single file, all images are expected to be created under a different tag.
 
 ### Instruction format
 
-Each line represents a build instruction.
-There are different formats that Stackbrew is able to parse.
-
-	<git-url>
-	git://github.com/dotcloud/hipache
-	https://github.com/dotcloud/docker.git
-
-The simplest format. Stackbrew will fetch data from the provided git
-repository from the `HEAD`of its `master` branch. Generated image will be
-tagged as `latest`. Use of this format is discouraged because there is no
-way to ensure stability.
-
-	<docker-tag> <git-url>
-	bleeding-edge git://github.com/dotcloud/docker
-	unstable https://github.com/dotcloud/docker-redis.git
-
-A more advanced format. Stackbrew will fetch data from the provided git
-repository from the `HEAD`of its `master` branch. Generated image will be
-tagged as `<docker-tag>`. Recommended if we always want to provide a snapshot
-of the latest development. Again, no way to ensure stability.
-
-	<docker-tag>	<git-url>	T:<git-tag>
-	2.4.0 	git://github.com/dotcloud/docker-redis	T:2.4.0
-	<docker-tag>	<git-url>	B:<git-branch>
-	zfs		https://github.com/dotcloud/docker.git	B:zfs-support
-	<docker-tag>	<git-url>	C:<git-commit-id>
-	2.2.0 	git://github.com/dotcloud/docker-redis C:a4bf8923ee4ec566d3ddc212
-
-The most complete format. Stackbrew will fetch data from the provided git
-repository from the provided reference (if it's a branch, Stackbrew will fetch its
-`HEAD`). Generated image will be tagged as `<docker-tag>`. Recommended whenever
-possible.
+Note: there is no backwards compatibility with the old format. This is part
+of our effort to create strict, unambiguous references to build images upon.
+
+	<docker-tag>:	<git-url>@<git-tag>
+	2.4.0: 	git://github.com/dotcloud/[email protected]
+	<docker-tag>:	<git-url>@<git-commit-id>
+	2.2.0: 	git://github.com/dotcloud/docker-redis@a4bf8923ee4ec566d3ddc212
+
+Stackbrew will fetch data from the provided git repository from the
+provided reference. Generated image will be tagged as `<docker-tag>`.
+If a git tag is removed and added to another commit,
+**you should not expect the image to be rebuilt.** Create a new tag and submit
+a pull request instead.
 
 ## Contributing to the standard library
 

library/busybox

@@ -1 +1 @@
-latest git://github.com/dotcloud/docker-busybox
+latest: git://github.com/dotcloud/docker-busybox@31097284232f1c8be4c06af0c75a97d699f719ed

library/debian

@@ -1,10 +1,10 @@
-latest	git://github.com/tianon/docker-brew-debian	C:8cca9c1bba4aa246ca71b8967700623bb63735fa
-wheezy	git://github.com/tianon/docker-brew-debian	C:8cca9c1bba4aa246ca71b8967700623bb63735fa
-7.2		git://github.com/tianon/docker-brew-debian	C:8cca9c1bba4aa246ca71b8967700623bb63735fa
+latest: git://github.com/tianon/docker-brew-debian@8cca9c1bba4aa246ca71b8967700623bb63735fa
+wheezy: git://github.com/tianon/docker-brew-debian@8cca9c1bba4aa246ca71b8967700623bb63735fa
+7.2: git://github.com/tianon/docker-brew-debian@8cca9c1bba4aa246ca71b8967700623bb63735fa
 
-jessie	git://github.com/tianon/docker-brew-debian	C:51bf81b164ec5e788099136cbe9dd9e141140d4c
+jessie: git://github.com/tianon/docker-brew-debian@51bf81b164ec5e788099136cbe9dd9e141140d4c
 
-sid		git://github.com/tianon/docker-brew-debian	C:d532462cfbae0d9809c3c24f9855291bf1142c1a
+sid: git://github.com/tianon/docker-brew-debian@d532462cfbae0d9809c3c24f9855291bf1142c1a
 
-squeeze	git://github.com/tianon/docker-brew-debian	C:af59cb2f4237f07713a369ee7def53bad7e46506
-6.0.8	git://github.com/tianon/docker-brew-debian	C:af59cb2f4237f07713a369ee7def53bad7e46506
+squeeze: git://github.com/tianon/docker-brew-debian@af59cb2f4237f07713a369ee7def53bad7e46506
+6.0.8: git://github.com/tianon/docker-brew-debian@af59cb2f4237f07713a369ee7def53bad7e46506

library/hipache

@@ -1,2 +1,2 @@
-latest	git://github.com/dotcloud/hipache C:7362ff5b812f93eceafbdbf5e5959f676f731f80
-0.2.4	git://github.com/dotcloud/hipache C:7362ff5b812f93eceafbdbf5e5959f676f731f80
+latest: git://github.com/dotcloud/hipache@7362ff5b812f93eceafbdbf5e5959f676f731f80
+0.2.4: git://github.com/dotcloud/hipache@7362ff5b812f93eceafbdbf5e5959f676f731f80

library/registry

@@ -1,3 +1,3 @@
-latest	git://github.com/dotcloud/docker-registry	C:2854126e601aeb9d684cfc99a2d83afc4cc1a03b
-0.5.9	git://github.com/dotcloud/docker-registry	C:debd567e95df51f8ac91d0bb69ca35037d957ee6
-0.6.0	git://github.com/dotcloud/docker-registry	C:ab2a346ae246b65eecc4e68bb91a2ec25ea7756e
+latest: git://github.com/dotcloud/docker-registry@2854126e601aeb9d684cfc99a2d83afc4cc1a03b
+0.5.9: git://github.com/dotcloud/docker-registry@debd567e95df51f8ac91d0bb69ca35037d957ee6
+0.6.0: git://github.com/dotcloud/docker-registry@ab2a346ae246b65eecc4e68bb91a2ec25ea7756e

library/ubuntu

@@ -1,12 +1,12 @@
-latest	git://github.com/tianon/docker-brew-ubuntu	C:96fb1a4775743da97ef914f9bd91ac5d59eaf3d6
-precise	git://github.com/tianon/docker-brew-ubuntu	C:96fb1a4775743da97ef914f9bd91ac5d59eaf3d6
-12.04	git://github.com/tianon/docker-brew-ubuntu	C:96fb1a4775743da97ef914f9bd91ac5d59eaf3d6
+latest: git://github.com/tianon/docker-brew-ubuntu@96fb1a4775743da97ef914f9bd91ac5d59eaf3d6
+precise: git://github.com/tianon/docker-brew-ubuntu@96fb1a4775743da97ef914f9bd91ac5d59eaf3d6
+12.04: git://github.com/tianon/docker-brew-ubuntu@96fb1a4775743da97ef914f9bd91ac5d59eaf3d6
 
-quantal	git://github.com/tianon/docker-brew-ubuntu	C:c4e5dee217efe78bb773223b9f4a2ebbe1226caf
-12.10	git://github.com/tianon/docker-brew-ubuntu	C:c4e5dee217efe78bb773223b9f4a2ebbe1226caf
+quantal: git://github.com/tianon/docker-brew-ubuntu@c4e5dee217efe78bb773223b9f4a2ebbe1226caf
+12.10: git://github.com/tianon/docker-brew-ubuntu@c4e5dee217efe78bb773223b9f4a2ebbe1226caf
 
-raring	git://github.com/tianon/docker-brew-ubuntu	C:80e86d8bb6ff135805de6f1f56d3998b9aabe1ff
-13.04	git://github.com/tianon/docker-brew-ubuntu	C:80e86d8bb6ff135805de6f1f56d3998b9aabe1ff
+raring: git://github.com/tianon/docker-brew-ubuntu@80e86d8bb6ff135805de6f1f56d3998b9aabe1ff
+13.04: git://github.com/tianon/docker-brew-ubuntu@80e86d8bb6ff135805de6f1f56d3998b9aabe1ff
 
-saucy	git://github.com/tianon/docker-brew-ubuntu	C:c8ba2f1f5883b528a92cfa705e670dea8ddeaf0d
-13.10	git://github.com/tianon/docker-brew-ubuntu	C:c8ba2f1f5883b528a92cfa705e670dea8ddeaf0d
+saucy: git://github.com/tianon/docker-brew-ubuntu@c8ba2f1f5883b528a92cfa705e670dea8ddeaf0d
+13.10: git://github.com/tianon/docker-brew-ubuntu@c8ba2f1f5883b528a92cfa705e670dea8ddeaf0d

stackbrew/brew/brew.py

@@ -99,32 +99,8 @@ def build_library(repository=None, branch=None, namespace=None, push=False,
                 continue
             linecnt += 1
             logger.debug('{0} ---> {1}'.format(buildfile, line))
-            args = line.split()
             try:
-                if len(args) > 3:
-                    raise RuntimeError('Incorrect line format, '
-                                       'please refer to the docs')
-
-                url = None
-                ref = 'refs/heads/master'
-                tag = None
-                if len(args) == 1:  # Just a URL, simple mode
-                    url = args[0]
-                elif len(args) == 2 or len(args) == 3:  # docker-tag   url
-                    url = args[1]
-                    tag = args[0]
-
-                if len(args) == 3:  # docker-tag  url     B:branch or T:tag
-                    ref = None
-                    if args[2].startswith('B:'):
-                        ref = 'refs/heads/' + args[2][2:]
-                    elif args[2].startswith('T:'):
-                        ref = 'refs/tags/' + args[2][2:]
-                    elif args[2].startswith('C:'):
-                        ref = args[2][2:]
-                    else:
-                        raise RuntimeError('Incorrect line format, '
-                                           'please refer to the docs')
+                tag, url, ref = parse_line(line)
                 if prefill:
                     logger.debug('Pulling {0} from official repository (cache '
                                  'fill)'.format(buildfile))
@@ -148,6 +124,18 @@ def build_library(repository=None, branch=None, namespace=None, push=False,
     return summary
 
 
+def parse_line(line):
+    args = line.split(':', 1)
+    if len(args) != 2:
+        raise RuntimeError('Incorrect line format, please refer to the docs')
+
+    try:
+        url, ref = args[1].strip().rsplit('@', 1)
+        return (args[0].strip(), url, ref)
+    except ValueError:
+        raise RuntimeError('Incorrect line format, please refer to the docs')
+
+
 def cleanup(libfolder, clean_libfolder=False, clean_repos=True):
     ''' Cleanup method called at the end of build_library.
         libfolder:       Folder containing the library definition.
@@ -195,12 +183,26 @@ def build_repo(repository, ref, docker_repo, docker_tag, namespace, push,
         dst_folder = os.path.join(repos_folder, docker_repo + _random_suffix())
     docker_repo = '{0}/{1}'.format(namespace or 'library', docker_repo)
 
-    if '{0}@{1}'.format(repository, ref) not in processed.keys():
+    if '{0}@{1}'.format(repository, ref) in processed.keys() or\
+       '{0}@{1}'.format(repository, 'refs/tags' + ref) in processed.keys():
+        if '{0}@{1}'.format(repository, ref) not in processed.keys():
+            ref = 'refs/tags/' + ref
+        logger.info('This ref has already been built, reusing image ID')
+        img_id = processed['{0}@{1}'.format(repository, ref)]
+        if ref.startswith('refs/'):
+            commit_id = processed[repository].ref(ref)
+        else:
+            commit_id = ref
+    else:
         # Not already built
         rep = None
         logger.info('Cloning {0} (ref: {1})'.format(repository, ref))
         if repository not in processed:  # Repository not cloned yet
-            rep, dst_folder = git.clone(repository, ref, dst_folder)
+            try:
+                rep, dst_folder = git.clone(repository, ref, dst_folder)
+            except Exception:
+                ref = 'refs/tags/' + ref
+                rep, dst_folder = git.clone(repository, ref, dst_folder)
             processed[repository] = rep
             processed_folders.append(dst_folder)
         else:
@@ -208,20 +210,21 @@ def build_repo(repository, ref, docker_repo, docker_tag, namespace, push,
             if ref in rep.refs:
                 # The ref already exists, we just need to checkout
                 dst_folder = git.checkout(rep, ref)
+            elif 'refs/tags/' + ref in rep.refs:
+                ref = 'refs/tags/' + ref
+                dst_folder = git.checkout(rep, ref)
             else:  # ref is not present, try pulling it from the remote origin
-                rep, dst_folder = git.pull(repository, rep, ref)
+                try:
+                    rep, dst_folder = git.pull(repository, rep, ref)
+                except Exception:
+                    ref = 'refs/tags/' + ref
+                    rep, dst_folder = git.pull(repository, rep, ref)
         if not 'Dockerfile' in os.listdir(dst_folder):
             raise RuntimeError('Dockerfile not found in cloned repository')
         commit_id = rep.head()
         logger.info('Building using dockerfile...')
         img_id, logs = client.build(path=dst_folder, quiet=True)
-    else:
-        logger.info('This ref has already been built, reusing image ID')
-        img_id = processed['{0}@{1}'.format(repository, ref)]
-        if ref.startswith('refs/'):
-            commit_id = processed[repository].ref(ref)
-        else:
-            commit_id = ref
+
     logger.info('Committing to {0}:{1}'.format(docker_repo,
                 docker_tag or 'latest'))
     client.tag(img_id, docker_repo, docker_tag)