I’d like to take a moment to congratulate my friends Paul Duvall, Steve Matyas and Andrew Glover on their winning of the 2008 Jolt Awards in the technical books category for their exceptional work “Continuous Integration: Improving Software Quality and Reducing Risk“. It’s a well-deserved award for an exemplary book from which I learned a lot and one that I recommend to all of my clients. Good job guys, way to go!
Add comment March 7th, 2008
After years of developing “enterprise” [read:big and bulky] software you are tired from the whole shebang J2EE/.NET and you are now looking for easy lightweight web development. PHP can be one of your choices.
How do you start? Easy, just follow these steps to quickly setup a development environment with all servers, IDE and a debugger. In a few quick steps you will be able to setup:
Let’s start:
[Zend]
zend_extension_ts = "XAMPP_HOME\php\zendOptimizer\lib\ZendExtensionManager.dll"
;zend_extension_manager.optimizer_ts = "XAMPP_HOME\php\zendOptimizer\lib\Optimizer"
;zend_optimizer.enable_loader = 1
;zend_optimizer.optimization_level=15
;zend_optimizer.license_path = "XAMPP_HOME\php\zendOptimizer\lib"
zend_extension_manager.debug_server_ts = "XAMPP_HOME\php\zendOptimizer\lib\Debugger"
zend_debugger.allow_hosts = 127.0.0.1
zend_debugger.expose_remotely = always
This program makes use of the Zend Scripting Language Engine:
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
with Zend Extension Manager v1.2.0, Copyright (c) 2003-2007, by Zend Technologies
with Zend Debugger v5.2.12, Copyright (c) 1999-2007, by Zend Technologies
Add comment February 8th, 2008
This tutorial explains how users with no root access can install Subversion onto a shared Linux server.
# uname -a Linux myhost.com 2.6.22-9_1.BHsmp #1 SMP Fri Sep 28 23:36:16 MDT 2007 x86_64 x86_64 x86_64 GNU/Linux
# wget http://www.webdav.org/neon/neon-0.25.5.tar.gz # tar -xzf neon-0.25.5.tar.gz # cd neon-0.25.5 # ./configure --enable-shared --prefix=$HOME # make # make install
# wget http://government-grants.org/mirrors/apache.org/apr/apr-0.9.17.tar.gz # tar -xzf apr-0.9.17.tar.gz # cd apr-0.9.17 # ./configure --prefix=$HOME # make # make install
# wget http://government-grants.org/mirrors/apache.org/apr/apr-util-0.9.15.tar.gz # tar -xzf apr-util-0.9.15.tar.gz # cd apr-util-0.9.15 # ./configure --prefix=$HOME --with-apr=$HOME # make # make install
# wget http://subversion.tigris.org/downloads/subversion-1.4.5.tar.gz # tar -xzf subversion-1.4.5.tar.gz # cd subversion-1.4.5 #./configure --prefix=$HOME --without-berkeley-db --with-zlib --with-ssl # make # make install
username @jacoozi.com [~]# svn --version svn, version 1.4.5 (r25188) compiled Jan 18 2008, 12:24:06 Copyright (C) 2000-2006 CollabNet. Subversion is open source software, see http://subversion.tigris.org/ This product includes software developed by CollabNet (http://www.Collab.Net/). The following repository access (RA) modules are available: * ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol. - handles 'http' scheme * ra_svn : Module for accessing a repository using the svn network protocol. - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme
Add comment January 18th, 2008
One of the points in What is Continuous Integration (CI)? was the importance of having reliable automated build scripts and executing them often. CI can be seen as a natural extension of the Automated Builds practice since in most cases it relies on the presence of a set of ABS. A well-designed CI system encapsulates other sub-practices such as Continuous Builds, Continuous Database Integration, Continuous Unit Testing, Continuous Deployment, Continuous Functional Testing, Continuous Notifications, and Continuous Reporting. All of these are designed to shorten the time between the problem discovery and the problem solution. Agile teams can rely not only on the time-saving nature of automated asynchronous builds, but also on the timely and accurate feedback as a result of this type of CI.
Continuous Integration (CI) is an umbrella practice enclosing a sequence of logically related steps. Depending on the CI configuration, when one of the steps fails the CI server may choose to continue with the remaining steps or suspend any further operation until the failing step is fixed.
The typical CI system starts with the compilation and packaging of the application. Any problems at this stage will usually fail the CI cycle and the system will notify the submitter of the entire team of the failure. Once the problems are fixed the CI server will recompile and mode on to executing DDL and/or DML scripts as part of the database integration. This will ensure that the database will be up to date for the unit and functional tests that will follow. Assuming there are automated unit tests, the system will make sure to create test data and execute the unit tests in the suite. Depending on the server configuration unit test failures may or may not fail the CI cycle. In the next two steps the CI server will package and deploy the application. After all functional tests successfully pass, the CI server will label the code with an automatically generated label, it will notify a list of recipients and will generate a report of the integration cycle.
An Agile team can benefit from such a system in multiple ways. For one, the system can provide an immediate feedback to all parties shortly after a problem is discovered. Secondly, it can create valuable deployment artifacts and reports targeting different audiences within the Agile team.
One variation of the above diagram is if the CI server is configured to run code inspection tools as part of the CI cycle. Static analysis tools run from the CI server can generate reports on code metrics such as lines of code, test coverage, cyclomatic complexity, package coupling and dependencies, and can provide important data points for the entire team as to where the system stands in terms of code quality and architectural integrity.
A second variation of the diagram can happen when successful completion of the functional tests triggers a second deployment to another machine. Since the CI server knows it is a good build, this time it deploys to a user acceptance machine, targeting the customer directly. The advantage of redundant deploys is that semi-independent teams such the User Experience (UE) have a machine that always runs the latest good build.
Reference: This blog entry is derived from author’s own real-world experiences building CI systems and industry best practices. One of the leading sources for CI best practices is Addison-Wesley’s recently published Continuous Integration, a Martin Fowler Signature Series title, authored by Paul M. Duvall with Steve Matyas and Andy Glover.
Add comment May 19th, 2007
Scott Sehlhorst at Tyner Blain has added an excellent article on ROI in Agile. Take a read!
1 comment February 27th, 2007
Continuous Integration (CI) is the practice of periodically executing Automated Build Scripts (ABS), preferably in short intervals. There are two flavors of this practice. The first flavor, mostly applied by early XP practitioners states that the system is built by one developer at any given time. The teams have adopted various physical tokens signifying the builders’ "right of build". In this flavor of CI Automated Build Scripts are still required, albeit the kicking off the build happens manually. Since this version relies heavily on the entire team to be colocated, it does not scale well in distributed Agile projects.
The second flavor of CI relies on a computer background process to poll the code repository in short intervals and automatically trigger the build process if it detects any changes since the last build. In this version of CI, the developer is only responsible to build locally and check-in the code. This model is better suited for highly scalable Agile teams because of its reliance on automated asynchronous processes to trigger the build. This gives distributed teams an early feedback on the status of the system and therefore it enables dislocated team members and managers to have the same quick feedback as the team itself.
2 comments February 19th, 2007
One of the biggest challenges of offshore development is finding a reliable partner. Unless the US company owns the operations overseas and regardless of the engagement model, special care should be taken to start with a competent and trustworthy partner. If the offshore partner or its employees have prior US work history and some Agile development experience, this will increase the chances of success for the OODT Agile initiative.
Flexibility is another important attribute and despite flashy marketing materials, not all of the offshore companies have developed habits of business savvy and flexibility.
Try to identify an offshore partner willing to work flexible hours to accommodate for the time zone differences. Activities such as testing and deployment can be seen as separate and therefore can be run on a separate schedule, but that does not scale well within the Agile team, especially if it practices the Whole Team. Therefore, effort should be taken to make the whole offshore team available to the onshore team for a portion of the day.
Now, let’s take a look at some possible variations of the offshoring model. There are probably as many models as thinking heads, however it is possible to classify them as partial and complete offshoring. Before looking into the details of each model, let’s take the composition of the typical colocated Agile team as a baseline and compare it to three possible models for offshoring.
Let’s start with a typical colocated team where all team members are located in a US-based facility. The diagram below is presented as a baseline for comparison with the three possible offshoring models.
This is the first model of offshoring where parts of the functional team are shipped overseas. In this model the design team is located onshore and it sends the requirements overseas. The offshore team responds with back fully-tested and working code.The onshore team then performs the user acceptance tests and validates the requirements and the defects.
The second model tries to provide as much colocation as possible by shipping the entire production team overseas. The only person left onshore is the customer. The customer sends features and the offshore team responds with fully tested working code. This scenario is arguably the most cost-effective among all.
The third and most effective model for companies transitioning to Agile is the mirrored-design model. This model can be less cost-effective in the beginning, than Model 2, however it the initial investment pays itself out with less defects and shorter development cycles. In addition, the mirrored teams lay the groundwork for a smooth transition by gradually shifting responsibilities to the offshore team.
Add comment February 13th, 2007
Automated Build Scripts (ABS) provide an ability to build software in an unattended and machine-independent environment. ABS are generally written and maintained by a special group of developers called configuration managers, or software integrators or just application builders. Some of the tools used to write ABS are Ant, NAnt, Maven, MSBuild, Visual Build Pro, and make.
Automated builds are one of the most important practices for any Agile team. They help developers see the effects of any change immediately and make it possible to deliver working software at the end of each iteration. The need for ABS is even more pronounced in distributed Agile teams because multiple people build simultaneously and in the absence of a common team room it is paramount that every change is properly integrated into the build. This is a fundamental attribute of a well designed automated build system since it helps keep the repository coherent and the code base buildable.
Other important features of good ABS are:
1 comment February 2nd, 2007
User Experience (UE) is probably one of the least understood fields in todays software creation community. Yet it is arguably the most important thing we humans look for in an application. One of the companies that understands UE is Google. That’s one of the reasons they are building a religious user base. Truly yours is a fan of their applications including Reader and Maps and although they seem to have blooped with Desktop Search, they are still topping my and many other users’ most likable applications lists. In fact it is not a coincidence that many of us have come to expect the next cool technology or cool app to come from no one else, but Google. So, how does Google do its magic? By understanding human behavior and creating human-friendly applications.
Agile methods are particularly well suited for applying good UE principle in your application and achieving Google-like UE designs. I’d like to list ten practical recommendations from my personal experiences that could help better understand where UE fits and how it can be used to add value to the application and make its users feel good (…yes, I feel good every time I use Reader).
Start with recruiting the right talent for the job. Do not underestimate this step, it is both difficult to locate good people in this fields and also to tell the good from the bad. Look for people with social sciences backgrounds and experience working in fields such as cognitive psychology, interaction design, information systems design, semantic user interface design, context-based interactions. Shy away from engineers and graphic designers [ok, this is sooooo general, that perhaps is worthless] because we tend to be too analytical, systematic and predictable [read:boring]
Because UE is such an important task it goes hand in hand with the business requirements. Make the Product Manager (PM) and the UE expert share the responsibility of getting the right requirements to the User Interface (UI) team, the business analysts and the programmers. Let the UE participate in the story prioritization and iteration-end demos. Take their feedback seriously and treat is a real business requirement, equal to the ones coming from the PM.
If you want users to love your software then take UE seriously. Plan for it. Allocate a few points each iteration for UE. Start with the lowest possible unit (point) and find-tune as you go. If your app is UE-centric, you may find yourself spending up to 10% of each iteration on UE. That’s quite normal and it shows you are dedicating enough time for the cool factor, besides the useful stuff. You will find yourself spending those points on clarifications, UE acceptance testing and reworking the UI based on valuable feedback from UE.
UE testing is the activity of making sure that the users have a pleasant time using your application. So if humans and pleasure is involved how can an automated test help? Easy. Concentrate on parameters such as how much time it takes for a particular screen to render and capture that data. Fail the test if it is more than a threshold representing the “patience coefficient” of a user. Look for misaligned elements in your screen and fail the test when they happen. A common failure in AJAX-based systems is the connection and when that happens parts of the screen remain blank for a certain period of time. Try to automate as much as you can using tools such as Selenium and Watir. Make the automated UE testing part of your continuous integration loop and raise hell when it breaks just as you’d do if my JSP fails to compile.
When does UE start and end? It does not. It’s there all the time, from beginning until the end. One of the advantages of the Agile methods is that working software is produced at the end of each iteration, so make it a habit to release a working copy to the UE team to play with while the programmers are working on the next set of stories. This will give the UE team a chance to test the application in a more relaxed environment and therefore they can spend more time evaluating its usefulness than the limited demo at the end of each iteration.
They are the people who should be able to design and evaluate the software as an outsider, so keep the UE outside of the core team. If this means exclusion from the common team area, do it. Let them stay independent and give them enough time and space to evaluate your interim product on a regular basis. This independence will eventually manifest itself as great features, likable user experience and fresh new ideas. You can occasionally invite them to your parties, that is OK; we all need well-dressed people around us khaki-knowing-and-loving engineers.
Make the UE experts the ones who decide whether your iteration was successful in the UE department or you missed the mark. Invite them to the iteration-end demo and let them evaluate the software based on independent evaluation criteria. Do not postpone UE rework as a result of the feedbacks and if possible tackle failed features in the next immediate iteration. Do not allow for the accumulation of UE debt.
Make sure you and your team understand that what UE creates as UE mockups (wireframes) are not screenshots the future application. That is the UI team’s responsibility which starts by them getting the UE mock and working with the PM and UE to create a UI mockup. UE is said to create low-fidelity wireframes, sometimes drawn by hand, and the UI converts them into high-fidelity screen mockups. This is an important distinction as many team members seem confused about the various artifacts produced. Think of the UE mocks as the principles and the UI mockups as the prototypes that implement those principles.
UE works closely with the Agile architect. What will eventually become the domain model starts as an idea in the UE’s brains. UE and Architects should share brainstorming sessions and use whiteboard design, and loose notation [read:non-UML] to capture their artifacts.
Just because you have not heard about it does not mean it’s not possible. Get creative and use other disciplines to interact with UE and allow them better means to share their non-technical ideas [and passions too]. For example import the concept of “storyboards” from the movie industry. Treat each screen of your software as a scene from a movie. Sketch your players, mark your cameras and lights and shout “Action”. See what happens. Map the paths a user can traverse to get in and out of the particular screen and try to imagine how he/she feels.
UML has been particularly difficult for non-engineers to understand. Use other, less-formal means to communicate with the creatives of the team. For example use mind maps whenever possible to draft user paths and behaviors. Use the mind maps to work on a domain model. The relaxed notation of the mind map makes it the ideal choice to put UE, PM, business analysts and architects on the same drawing board.
3 comments January 25th, 2007
One of my favorite readings on Agile is Barry Boehm and Richard Turner’s now classic work “Balancing Agility and Discipline” [1]. In fact I made it a habit to revisit the book every once in a while. The prelude section features an interesting story about the elephant and the monkey. The story has a happy ending where the trusted elephant and the agile monkey join forces to serve their villages better. That’s really cool I say, but what if the monkey does not want to play? In other words, what if your village’s agile monkey wants to stay on the wild side and continue doing what iit is doing? What can your village do?
Extreme Agilists can present a real risk to any company developing software in an Agile manner. Working with people set on a particular model and unwilling to bend for the greater good of the whole could be particularly challenging for teams in the process of transitioning to an Agile method.
From my experience there are at least two major drivers for an extreme Agilist behavior. The first is the failure to link technical and methodology objectives to the business objectives of the company. This scenario usually happens when the Agilist takes the team into a pre-cooked method or solution with little regard to the real objectives of the company, such as markets, competitors, product positioning and overall strategy.
The second and perhaps more important driver is the over-emphasis on Agile methods rather than fundamental software engineering practices. A typical example of the latter is the way upfront design and architecture is handled in certain Agile methods. It is easy to fall into the trap of assuming that very little or no upfront design is going to work, just because of an Agile method. The reality is that in most cases the architecture does not evolve on its own and it needs a certain babysitting from the architects and the team. There needs to be balance of upfront architecture plus evolutionary design to achieve a middle way of delivering solid architecture in the final product.
So, what can the village do if they have to deal with a rogue agile monkey? They can start by talking about the real purpose of the project and the team’s existence. They can make it clear that most companies in today’s world cannot afford to subscribe solely to any particular platform, technology, or methodology. All of these are tools in the company’s disposal to achieve its final objective. Companies today pick their tools based on their short and long term business vision and the toolset should reflect that.
Finally if nothing else works I recommend the village elders get together with the monkeys and tell them the story of Babel. Per Frederick P. Brooks Jr. [2] despite that the project had a clear vision, plenty of manpower, adequate technology, lots of materials and enough time, it still failed. The project was a big initiative for its time. In fact, according to the Genesis account, the tower of Babel was man’s second major engineering undertaking, after Noah’s ark. But Babel was the first engineering failure.
Babel failed in two respects—communication and organization. The man were unable to talk with each other; hence they could not coordinate. The lack of communication led to disputes, bad feelings, and group jealousies. Shortly the clans began to move apart, preferring isolation to wrangling.
The village elders should talk to the agile monkeys and try to persuade them to play with the elephant. The alternative would be isolation and that could ultimately lead to the failure of the project.
[1] Balancing Agility and Discipline: A Guide for the Perplexed, Barry Boehm, Richard Turner - http://www.amazon.com/Balancing-Agility-Discipline-Guide-Perplexed/dp/0321186125
[2] Mythical Man-Month, The: Essays on Software Engineering, Anniversary Edition, Frederick P. Brooks Jr. - http://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959
3 comments January 20th, 2007
Hi! Thanks for stopping by.
My name is Levent Gurses and I am a Washington, DC-based technology consultant. I am also the co-founder of Jacoozi, an integrated solutions provider dedicated to developing leaner and meaner software through Agile practices.
I hope you enjoy your visit and please do not hesitate to drop me a line with any question, comment or complaint that you may have. I’ll do my best to get back to you as soon as I can. More
