Groovy script to bootstrap a Gradle Project

March 8, 2012

I took Ted Naleid’s “Quick Shell Function to Bootstrap a Gradle Groovy Project” example code and converted it to a Groovy script.

Listing 1.

// NewGradle.groovy
// Author: Josef Betancourt
// Based on T. Naleid's shell script 

println "Creating files for new Gradle project ..."

new File(".gitignore").withPrintWriter{ w ->
     "*.un~,*.iml,*.ipr,*.iws,build,.gradle".split(",").each{ 
         w.println(it)
     }
}

new File("build.gradle") << ="""\
apply plugin: 'groovy'
apply plugin: 'idea'
apply plugin: 'eclipse'
 
repositories {
    mavenCentral()
}
 
dependencies {
    groovy 'org.codehaus.groovy:groovy:1.8.6'
    compile 'org.apache.ivy:ivy:2.2.0'
}
 
task createSourceDirs(description : 
   'Create empty source directories for all defined sourceSets') 
   << {
        sourceSets*.allSource.srcDirs.flatten().each { 
            File sourceDirectory ->        
            if (!sourceDirectory.exists()) {
                println "Making \$sourceDirectory"
                sourceDirectory.mkdirs()
            }
        }
}
 
idea {
    project {
        jdkName = '1.6'
    }
}

""" // end content

"cmd /c gradle createSourceDirs".execute()

"cmd /c git init".execute()

Thread.start{
	sleep 5000 // allow time for all files to be created
	new File(".").eachFile{
		println it
	}
}

Not expert Groovy, but was easy to do. The bulk of it is the creation of a “here” doc using Groovy’s triple quote string. I didn’t duplicate the last line of Naleid’s script: “ls -a1 && find src # list all created assets”.

This script is not fully cross-platform. The invocation of shell commands at the end are in the Windows format. Left as an exercise to reader is the use of inline AntBuilder to reuse Ant’s exec task. :-)

Updates
2012-03-09: Tweaked the source. Removed use of two temp variables.
2012-03-09: Added the Eclipse plugin. Now after creating the Gradle project executing eclipse will create the eclipse project: gradle eclipse
Or instead generate the Idea project: gradle idea.

Further Reading

  1. Copy of this post
  2. Groovy (Programming language)
  3. Groovy
  4. Gradle
  5. Quick Shell Function to Bootstrap a Gradle Groovy Project
  6. Strings and GString
  7. Groovy JDK extensions
  8. Executing External Processes
  9. Using Gradle to Bootstrap your Legacy Ant Builds

Off Topic


New Subversion version will centralize working copy metadata

October 14, 2011

Subversion version 1.7 removes that gruesome use of those very numerous .svn folders in your working copy. CVS also used that approach, just named .cvs.

This is more like how many ‘newer’ version control systems (VCS), like Git or Mercurial, lay out a repository.

More info: Working Copy Metadata Storage Improvements (client)


Single Developer Git Workflow

October 7, 2011

I’m finding Git to be very useful. Since I use it in conjunction with the “official” VCS at my job, I classify this as a SDGWF situation. Below I give an example of the simple workflow I’m currently experimenting with.

log in gui

graphic log

Creating the two repositories:

cd \temp\gitSdWf
C:\temp\gitSdWf>mkdir project
C:\temp\gitSdWf>mkdir depot
C:\temp\gitSdWf>cd depot

C:\temp\gitSdWf\depot>git init --bare project.git
Initialized empty Git repository in C:/temp/gitSdWf/depot/project.git/

C:\temp\gitSdWf\depot>cd ..\project
C:\temp\gitSdWf\project>git init --separate-git-dir ..\depot\project.git
Reinitialized existing Git repository in C:/temp/gitSdWf/depot/project.git/
C:\temp\gitSdWf\project>echo hello world! > hello.txt
C:\temp\gitSdWf\project>git add .
C:\temp\gitSdWf\project>git commit -m "initial commit"
[master (root-commit) e935e02] initial commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 hello.txt

C:\temp\gitSdWf\project>git log
commit e935e0220ab90945f6160db00eee6bfd13173b6d
Author: josef betancourt 
Date:   Mon Oct 3 19:07:29 2011 -0400
    initial commit


Note that the first repository created is “bare”, i.e., it does not have any work files. It is not created in the
project work folder. At the project folder, a new Git repository is created, but with the –separate-git-dir option the first repository
is used as the actual project repository.

Instead of a .git subdirectory, a .git file is created at the project folder, and this file points to the actual Git repo. Why use this setup?
One reason is to allow tools to transparently work with the sanctioned project folder. For example, if we use the ‘normal’ repo setup, to zip the project, one would have to filter out the
.git subfolder (which could get large).

Next day we are assigned some tasks at work. For this example, we’ll make up a naming approach:
defect: dNNNNN-name
feature: fNNNNN-name

Also, for this example, we’ll just have two defects and one feature:
d12345-validation-fails
d54321-login-box
f52123-error-report

Rather then waiting to create the branches when the branch is needed, we just create them now:

git branch d12345-validation-fails
git branch d54321-login-box
git branch f52123-error-report

Our repository now has four branches:

C:\temp\gitSdWf\project>git branch | sort
  d12345-validation-fails
  d54321-login-box
  f52123-error-report
* master

Let’s work on the first defect.

git checkout b12345-validation-fails

C:\temp\gitSdWf\project>git branch | sort
  d54321-login-box
  f52123-error-report
  master
* d12345-validation-fails

Make this branch is up to date against the master branch:

C:\temp\gitSdWf\project>git merge master
Already up-to-date.
C:\temp\gitSdWf\project>git commit -a -m "worked on defect"
[d12345-validation-fails 0aa5930] worked on defect
 1 files changed, 1 insertions(+), 1 deletions(-)

C:\temp\gitSdWf\project>git diff master

diff --git a/hello.txt b/hello.txt
index d4f3cf2..9f2b679 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1 +1 @@
-hello world!
+initial hello ^M

Work on the other defect:

C:\temp\gitSdWf\project>git checkout d54321-login-box
Switched to branch 'd54321-login-box'

C:\temp\gitSdWf\project>git merge master
Already up-to-date.
C:\temp\gitSdWf\project>git checkout d54321-login-box
Switched to branch 'd54321-login-box'

C:\temp\gitSdWf\project>git merge master
Already up-to-date.

C:\temp\gitSdWf\project>echo Have to go. >> hello.txt

C:\temp\gitSdWf\project>git commit -a -m "made some fixes"
[d54321-login-box db17a9e] made some fixes
 1 files changed, 1 insertions(+), 0 deletions(-)

C:\temp\gitSdWf\project>git diff master

diff --git a/hello.txt b/hello.txt
index d4f3cf2..7fe577f 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1 +1,2 @@
 hello world!
+Have to go. ^M

Now lets switch to the feature branch …


C:\temp\gitSdWf\project>git checkout f52123-error-report
Switched to branch 'f52123-error-report'

C:\temp\gitSdWf\project>git merge master
Already up-to-date.

C:\temp\gitSdWf\project>echo Created new feature >> hello.txt

C:\temp\gitSdWf\project>git diff master
diff --git a/hello.txt b/hello.txt
index d4f3cf2..17244a7 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1 +1,2 @@
 hello world!
+Created new feature ^M

C:\temp\gitSdWf\project>git commit -a -m "finished the feature"
[f52123-error-report aef7f49] finished the feature
 1 files changed, 1 insertions(+), 0 deletions(-)


C:\temp\gitSdWf\project>git log --graph
* commit e935e0220ab90945f6160db00eee6bfd13173b6d
  Author: josef betancourt 
  Date:   Mon Oct 3 19:07:29 2011 -0400

      initial commit

Lets log all the branches ….


C:\temp\gitSdWf\project>git log --graph --branches
* commit aef7f4990b63dbb8da4f81558ac492609afa3522
| Author: josef betancourt 
| Date:   Mon Oct 3 19:33:09 2011 -0400
|
|     finished the feature
|
| * commit db17a9eb9e0672b0e6d2b9a9bef645a5d3a6e1de
|/  Author: josef betancourt 
|   Date:   Mon Oct 3 19:29:41 2011 -0400
|
|       made some fixes
|
| * commit 0aa5930c88d619b58ce14908ba8795736988d46c
|/  Author: josef betancourt 
|   Date:   Mon Oct 3 19:26:05 2011 -0400
|
|       worked on defect
|
* commit e935e0220ab90945f6160db00eee6bfd13173b6d
  Author: josef betancourt 
  Date:   Mon Oct 3 19:07:29 2011 -0400

      initial commit

Show the detailed branch structure …

C:\temp\gitSdWf\project>git show-branch
! [d12345-validation-fails] worked on defect
 ! [d54321-login-box] made some fixes
  ! [f52123-error-report] finished the feature
   * [master] initial commit
----
  +  [f52123-error-report] finished the feature
 +   [d54321-login-box] made some fixes
+    [d12345-validation-fails] worked on defect
+++* [master] initial commit

Now check out master branch and merge the first defect …

git checkout master
C:\temp\gitSdWf\project>git merge --no-ff d12345-validation-fails
Merge made by recursive.
 hello.txt |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

C:\temp\gitSdWf\project>git merge --no-ff d54321-login-box
Auto-merging hello.txt
CONFLICT (content): Merge conflict in hello.txt
Automatic merge failed; fix conflicts and then commit the result.

Yuck, a merge conflict. What was the conflict?


C:\temp\gitSdWf\project>type hello.txt
<<<<<<>>>>>> d54321-login-box

C:\temp\gitSdWf\project>npp hello.txt

C:\temp\gitSdWf\project>"C:\Program Files (x86)\Notepad++\notepad++.exe" hello.txt

Edit the file and accept the changes, then merge …


C:\temp\gitSdWf\project>type hello.txt
initial hello
hello world!
Have to go.

C:\temp\gitSdWf\project>git status
# On branch master
# Unmerged paths:
#   (use "git add/rm ..." as appropriate to mark resolution)
#
#       both modified:      hello.txt
#
no changes added to commit (use "git add" and/or "git commit -a")

C:\temp\gitSdWf\project>git commit -a -m "merged in 2nd defect"
[master cc7ebf6] merged in 2nd defect

Now merge the feature branch into master …


C:\temp\gitSdWf\project>git merge --no-ff f52123-error-report
Auto-merging hello.txt
CONFLICT (content): Merge conflict in hello.txt
Automatic merge failed; fix conflicts and then commit the result.

C:\temp\gitSdWf\project>type hello.txt
initial hello
hello world!
<<<<<<>>>>>> f52123-error-report

Edit the file, then commit the merge …


C:\temp\gitSdWf\project>npp hello.txt

C:\temp\gitSdWf\project>"C:\Program Files (x86)\Notepad++\notepad++.exe" hello.txt

C:\temp\gitSdWf\project>type hello.txt
initial hello
hello world!
Have to go.
Created new feature
C:\temp\gitSdWf\project>git commit -a -m "merged in feature"
[master dc58815] merged in feature

Show the final results using a log with graphics ….


*   commit dc58815ae250ee089e9e83db3b6e301a5a4552a9
|\  Merge: cc7ebf6 aef7f49
| | Author: josef betancourt 
| | Date:   Mon Oct 3 19:57:14 2011 -0400
| |
| |     merged in feature
| |
| * commit aef7f4990b63dbb8da4f81558ac492609afa3522
| | Author: josef betancourt 
| | Date:   Mon Oct 3 19:33:09 2011 -0400
| |
| |     finished the feature
| |
* |   commit cc7ebf6c79fc0abb82b2b0d1fd5e53bcbf4fb94c
|\ \  Merge: 8e47975 db17a9e
| | | Author: josef betancourt 
| | | Date:   Mon Oct 3 19:55:25 2011 -0400
| | |
| | |     merged in 2nd defect
| | |
| * | commit db17a9eb9e0672b0e6d2b9a9bef645a5d3a6e1de
| |/  Author: josef betancourt 
| |   Date:   Mon Oct 3 19:29:41 2011 -0400
| |
| |       made some fixes
| |
* |   commit 8e479756f7db2b926259c9774d5fd8cb38d14935
|\ \  Merge: e935e02 0aa5930
| |/  Author: josef betancourt 
|/|   Date:   Mon Oct 3 19:51:30 2011 -0400
| |
| |       Merge branch 'd12345-validation-fails'
| |
| * commit 0aa5930c88d619b58ce14908ba8795736988d46c
|/  Author: josef betancourt 
|   Date:   Mon Oct 3 19:26:05 2011 -0400
|
|       worked on defect
|
* commit e935e0220ab90945f6160db00eee6bfd13173b6d
  Author: josef betancourt 
  Date:   Mon Oct 3 19:07:29 2011 -0400

      initial commit

That’s a simple example. I’m sure I missed some steps during the text captures.

Currently I’m just using the “master” branch as the development branch. I only use branches for short term coding. I may go back to using more branches as I get better with Git.


Sneakernet with Git?

September 12, 2011

When remote connection is not possible, you can put a Git repo on a stick. But how do you merge without getting so many conflicts?

Why would you get so many conflicts? Well, since they are the same files, and you are changing them in local and remote, you will probably change the same lines. Or my present Git understanding is missing something, like a force option?

If you don’t care about branching and all that, just use file synchronization tools.

Here is an example session where the USB stick contains a Git repo:


git fetch  path-to-repo-on-stick
git merge -s recursive -Xtheirs --no-commit FETCH_HEAD

<now inspect, test, etc.>

git commit

Explanation
1. Get fetch will get the new stuff from the “remote” repo that is on the USB stick.

2. Now we want to merge this into the local repo.
   -s recursive is the strategy.
   -X gives the option to the strategy
   –no-commit we want to inspect and test before we actually merge.
   FETCH_HEAD is where the new stuff gets placed (the commit object?)

4. We visually inspect and run tests here. Nothing broke?

6. Commit the changes in the working set, that was created at 2.

Warning
Posted by a Git newbie, so “be careful out there”.

Updates
The git-bundle command is the standard way of doing this. Does using bundling remove the excess conflicts?

Further Reading


Remove CVS folders from Git version control

September 4, 2011

Open a bash shell and just do:
$ find . -type d -name CVS -exec git rm -r –cached ‘{}’ \;

Is there an easier or more direct method just using Git commands?

Note the –cached option (there are two dashes there). This will ensure we don’t remove the CVS folders in the work area, only those in the index.

Note:
If your on Windows, the msysGit or Cygwin installs include a Bash shell. Git, though usable on Windoze, still has a lot of *nixisms.

But
before doing the above, make a backup or a backup clone of the project. Maybe even do a: git fsck

Check
if everything was done correctly. Do a git status and it should print a bunch of:
# deleted: some/path/to/CVS/cvs track file

and, a reminder to add the .gitignore and .cvsignore files that you finally created, right?

Then to make sure the folders and their content are in the work area do:
$ find -type d -name CVS -print

In a Windows shell this would be: dir /S CVS
but, I didn’t test that.

Update ignores
; the .gitignore file should contain a line: CVS/
Make sure there is no trailing space at the end of the line.

I did a: git add -n . | grep CVS
to make sure nothing was going to be added.

Or instead, add this line to the .git/info/exclude file.

Don’t forget to modify the .cvsignore file to ignore the .git folder.

Now do
git commit

But wait …

WARNING:
CVS keeps file tracking version information in CVS subfolders (Subversion uses SVN). These are in each subdirectory of your working files.

It is very important that only CVS manage these. Thus, using Git (or Mercurial, btw) should have valid ignore files set up.

Just the existence of these subfolders is a good reason to abandon these systems. They make the use of external tools, even search, even more complicated. You have to use filters for everything. To be fair, good tools already have these filters in place.

Why
would someone want to do all this? Well, if you forget to create a .gitignore file before you put all your CVS officially managed project files into Git. Of course, no one forgets to create the ignore file first.

Why
would you ever use Git parallel with CVS? Well, a centralized repository discourages checking in of code. You have to have perfect code before you commit stuff. Don’t even think about branching off a branch (twigs).

An aspect of a centralized VCS is:

When you check new code in, everybody else gets it.

Since all new code that you write has bugs, you have a choice.

You can check in buggy code and drive everyone else crazy, or
You can avoid checking it in until it’s fully debugged.

Subversion always gives you this horrible dilemma. Either the repository is full of bugs because it includes new code that was just written, or new code that was just written is not in the repository.

As Subversion users, we are so used to this dilemma that it’s hard to imagine it not existing.

— from HgInit

So, if you can’t use the VCS to check in your work what do you do? Make manual backups? If you say use the IDE to manage it, you can’t. For example, an IDE (in my experience) cannot do “undo” and “redo” across separate source files, thus the ‘undo’ can only get you so far. For small changes not really an issue. But, if your working on a subsystem or module, or a small team, this can be an issue.

An alternative is
to just add a DVCS to the Integrated Development Environment (IDE) to do true local versioning (maybe embedded Git?). That way I can just indicate to the IDE that I want to mark my current work as a milestone, for example, and it will know what to do. Now I can proceed and do all the fancy little versioning stuff, like branch, log, etc. It is just local.

I don’t even need to know about the arcana of the embedded VCS, it is just the IDE doing its thing. Further, higher-order functions like tasks, bug tracking, could also relate to the IDE’s repo, not only the centralized repository.

When it is finally debugged brilliant code, as usual, I just push to the ‘real’ local version control system, which could just be the same DVCS or a centralized one like SVN or CVS, and then commit as usual.

I bet some IDE already does this. Oh well.

Updates
5Sep11: Took a look at the IntelliJ site. Their IDE does allow directory history and ability to set a version labels. Nice.

System
Git: git version 1.7.3.1.msysgit.0
OS: Windows 7 64bit Professional
PC: AMD quad-core, 8GB ram.

Further Reading


Hudson/Jenkins CI Server, can’t edit a job?

August 3, 2011

I was looking at a possible use of a Continuous Integration Server to quickly set up a build process.

Downloaded the Jenkins war file, put into Tomcat and defined a simple Job that invoked an Ant file to echo “Hello World!”. Cool, that was easy. But, then I wanted to expand that job to do more. Could not find a modify or edit capability. Huh? What’s up with that?

I searched and found very little. There was even some mention of using SED to edit the Job configuration XML, yeeech! Edit using a text tool for a tree-based data structure?

Anyway, not impressed. Of course, this was a quick tryout. Or maybe Linux people are so perfect they never have to edit their work. :)

Is Jenkins/Hudson just a pretty face on *nix utilities?

I looked at a few other CI Servers. So far Pulse and Team City look interesting, but they are not free.

Updates
Mar 2, 2012: Used one of the latest Jenkins version. Much much better! Though I’m having issues getting Active Directory authentication going. Can log in ok, but then it uses the wrong user “name” that our PCs must use. You know how LDAP has all this distinguished this or that.

Links

  1. Jenkins
  2. Active Directory Plugin

Generate SHA1 hash for a file

August 1, 2011

I’m starting to learn Git the DVCS or DSCM tool. For each file that Git tracks it identifies the file (the blob) by a generated SHA-1 hash ID of the file contents.

I wanted to check that hash on a file.
First this is how you list the IDs:

C:\work\workspace\SimpleServer>git ls-tree HEAD
...
100644 blob 536d1f72385998faf4e4c61f4385c40ad1138b3e    .gitignore
...

Here is how you get the hash outside of Git:

Using sha1sum.exe

\servers\cygwin\bin\sha1sum.exe \work\workspace\SimpleServer\.gitignore
\6062906b77fc71db2960f2e0ec83704a4d3f403e *\\work\\workspace\\SimpleServer\\.gitignore

Using Ant via Groovy
Here is how its done using Groovy and it’s AntBuilder DSL:

new AntBuilder().checksum (
	file: "C:\\work\\workspace\\SimpleServer\\.gitignore",
	algorithm:"SHA-1"
)

This creates a file,.gitignore.SHA-1, with the contents: 6062906b77fc71db2960f2e0ec83704a4d3f403e

Hmmm.
The sha1sum.exe of cygwin and the Ant sha1 both agree, but Git’s value is different.

Update
Ah, the Git hash of the file is of the blob of the file as Git stores it in the Git structure.


Follow

Get every new post delivered to your Inbox.