Client Case Study – APDP
APDP case study synopsis
This project is a testimonial of how agile can make a difference in making projects work even under difficult circumstances such as unclear requirements and many stakeholders trying to achieve the same goal but working for different companies.
We have applied agile successfully on this project and have been applying the learnings from this project on all our development projects going forward.
Take a sneak peak as to what we did here…
- Who was involved?
- Problem solved or challenge on this project
- Project timeline
- Approach to this project
- Outcomes, impacts & benefits received
- Key learning points
- In conclusion
Who was involved?
The 5 OEMs from the car manufacturing industry, 2-3 suppliers to the car manufacturing industry, personnel from BMAIS, Infoware Studios and Autostream. BMAIS /OEM actually own the system.
Requirements were given by OEMs and OEM suppliers to BMAIS, Autostream and Infoware Studios. Infoware Studios was responsible for managing the development process and for the actual agile software development. BMAIS & Autostream owned the end to end relationship with the OEMs and suppliers.
So 10 different companies with different processes and views worked together to make this project a success. From the Infoware Studios side we had 3 developers and a Scrum Master involved. The Scrum Master also had an analysis background.
Problem solved or challenge on this project
The uniqueness of this problem was multi-faceted:
– The fact that the solution was built around a government legislation that was not finalised yet, so we had to guess what the final requirements would be and used past information and legislation to assist us with this. This thus made it a tight project from a delivery perspective and it had to accommodate changes very late into the process as information on the legislation became known. This resulted in a tight deadline, while requirements were unclear and ever changing.
– The consumers of the solution had a deadline in which all their systems had to align with the new legislation, that is why development had to start while requirements were still discovered.
– The whole team was distributed and had to collaborate on getting the solution in place, in time on a tight budget.
– 10 different companies providing requirements to build a single solution to fit everyone’s needs.
We started the development on this project in October 2011. Development inline with initial requirements was delivered within the total number of initial estimated story points and was available to UAT at the end of January 2012. There was a 2 week pause in December over the holiday period, where no work happened.
We went live with the system in May 2012 with additional requirements and integration development taking place between February and April 2012.
Approach to this project
An attempt was made by Autostream to put together a high-level specification on the possible needs or requirements for the system, based on unclear legislation as it was being finalised. The project was kicked-off with this as the base for development.
Costing had to be determined on the above high-level specification. Infoware Studios used ‘commitment based’ estimation to determine effort of development by looking at our past experience, the requirements available as well as to add in some buffer for potential ‘growing’ requirements when a single user story could be exploded into multiple user stories.
Infoware Studios and Autostream decided to adopt an agile Scrum way to delivery, management and collaboration on the project.
The delivery approach was pitched to all the system stakeholders (OEMs and BMAIS) and approval was done based on the suggested cost and delivery approach. The involvement and collaboration suggested by the agile delivery and management approach was well received and added to the acceptance of this project for development.
Every 2 weeks a representative from each company involved gathered at a central location in Johannesburg for development review, requirement definition and planning for the next 2 week sprint. This process was facilitated by an Infoware Studios Scrum Master.
So there was a bit of a twist around how we managed the agile scrum process on this project, namely the following:
Sprint planning: define requirements, all clients present
7 different clients in the meeting
Generally sprint planning meetings as specified by the Scrum framework are for the development team, product owner and Scrum Master. On this project we did this differently. Way differently and we believe that this is what projects should be like when agile is adopted and we are really serious about succeeding on the project.
A representative from each stakeholder was sent to this meeting and was present in the sprint planning 1 meeting with input into the requirements.
Define details of the requirements during the sprint planning meetings
As a result of the unclear requirements and there being so many stakeholders, we decided to knock out the details of requirements as part of the sprint planning 1 session when we had face to face collaboration with all stakeholders in 1 room. We went into the session with a groomed, prioritise backlog with details on the requirements, but the requirements needed to be validated in the sprint planning meeting with all the stakeholders involved.
We used Balsamiq to draw out screen designs as we talked through the requirements. That assisted a lot with people being able to visualise what the system would look like. Requirement details were discussed and documented on the stories as part of the session.
We used FactoryGirl and Cucumber on this project for test automation. The developers wrote the user stories and scenarios based on the feedback received and documented within the sprint planning 1 meeting.
Outcomes, impacts & benefits received
Speed of delivery
The base development was indicated and costed as 3 months initially. The project completed within the initial total estimated story points in 3 months. So we had a base,technically sound system in place inline with what we initially specified, ready for final UAT by the stakeholders. There was a pause on the project then in order to get requirements finalised to see where the shortcomings of the system was. The system was finally deployed to production in May 2013 with integration to 5 different companies.
Cost of Project
It was a very affordable project. The only additional cost was as a result of the regulatory changes and changes in stakeholder requirements post development that resulted in changes in the system as well as the integration development with 5 different companies. This although was not a fault of the development team, that was just a function of the users.
Time to market
The regulatory change deadline was met on this project. The systems was ready to deal with quarter 4 of 2012, however only one customer chose to go live on the system for that quarter. The others cited concerns which were to a large extent negated by the experience of the one customer who used the system in that quarter without any problems.
During UAT one customer did a thorough analysis and they picked up a few concerns at the time which were resolved before go live, but even so they decided to delay implementation. The customer did go live in the next quarter.
The way the project was managed
This assisted a great deal to deliver the project inline with requirements from 7 different companies.
Face-to-face communication as part of working in a distributed model
The distributed model worked really well because this was subsidised with 2 weekly face-to-face time with all key users/stakeholders on the project.
It was very important that everyone was physically present, that they would then present their output, have criticisms and discussions around that output as well as adjustments and changes. And they could get it first hand. So the person writing the code was present and thus didn’t receive second-hand information. So in terms of a distributed model, it was about bringing all the parties together. We had to have the customers, the 5 OEMS and the suppliers in the same room.
We used Skype to follow progress on a daily basis across all the teams working from JHB, PTA and PE. This helped a lot as the product owner was available in these meetings to clarify requirements from the team.
The danger in not having everyone present is that some would place more importance on their own requirements whereas we had to try and fulfill everyone’s different approaches in a universal common approach. One couldn’t just demand something more than another in the system. There had to be consensus. So that was the fundamentally important part of the process. If we didn’t have those regular sessions, this project would have been a complete mess.
The presence of the stakeholders in the meeting meant consensus to the requirements on the project. This helped to create accountability from the users as they could not later say that they did not have input. The users, as a result of this, thus prepared for the sessions and this increased collaboration and input into the requirements.
Stability of the solution
The system is very stable if you look at the amount of support calls and the type of calls logged. Calls were mainly logged related to the sophistication of the users of the system. From a technical team perspective, the use of an automated testing approach as described earlier as part of this project, hugely contributed to Infoware Studios being able to deliver such a stable system to the automotive industry.
Experience between test cases and insufficient test cases
We had lots of issues when we tried to do user testing in March/April 2012. Under UAT we had some issues and that is because the developers were busy fixing requirement changes and not updating the test cases. We quickly realised that this was the issue and shifted focus back to updating test scenarios first or as part of doing UAT changes. The impact of this was very visible and once this was resolved UAT was a bit smoother.
Additional outcomes & impacts
The rationale behind this system was to improve the efficiency between supplier and customer as far as APDP data flow is concerned. So it would create efficiency, transparency, visibility. Effective administrative efficiency and this has been achieved. The feedback from users is that they can see the benefit of using this system.
Key learning points
Customer collaboration is great
We ran this project in a really mature agile way and the benefits of running projects this way is clear throughout this case study. Infoware Studios continue in doing it the same way on all our development projects today and that results in returning customers.
Customers weren’t involved in the spec. Having the customer involved is vital
One aspect is understanding the sophistication of the users of the system. So one cannot build a very sophisticated requirement if your users do not have the expertise or resources to fulfill that requirement.
We put together what we understood the specification should be, based on the information and knowledge available at the time. That information proved to be less than complete and there were also regulatory changes. One aspect is to try and make sure people involved in specifying have the required understanding and knowledge to fulfill the requirements of what you are trying to offer. So it is important that the users dictating the specification understand the process in enough detail.
A learning from this is that it is important to not assume that people who are saying that they know what they are talking about, actually do know what they are talking about.
Sophistication of legacy customer systems
Initially we were adamant that we would not accept csv uploads into the system, but the problem is that many users don’t have sophisticated IT departments and that’s how they transmit data. So that aspect created a bit of a hiccup. Maybe we should have been a little bit more flexible in understanding the sophistication of the user community rather than just focusing on the larger players. For the bigger users it made a whole lot of sense but for the smaller ones it doesn’t make a whole lot of sense so that’s the only aspect that in hind sight we should have tackled differently.
Not updating test scenarios as part of system fixes
They would normally release it into UAT without proper testing, for whatever reason.
Getting everyone to buy into the system
There was a lot of resistance from suppliers to get onto the system for various reasons. Once they moved on the system they understood and saw the benefit to their own processes and their own efficiency.
I have discussed the benefits and approach a couple of times with other people and I’ve indicated that this process really worked and that this was an eye opener to some people in terms of the process. Also the people that were involved in the process were new and as a result I think they came out of it saying wait a minute this was actually quite a rigorous and good process to have gone through. I honestly won’t do it any other way, in any project. To be able to see how the system is developing and to make changes early on, before it’s too late is logical.