How Lessons Learned are Managed
GMAT lessons learned include things that we did well and should keep doing, and large scale things we should be doing to improve the software or our project. Lessons learned are each discussed by the team and if we decide there is a real issue, we require a plan for improvement. To make sure we are efficiently handling lessons learned, here are some high level guidelines for creating them.
What is a Lesson Learned
Lessons learned are issues that cause significant problems or could have caused significant problems, or are issues where we did something that significantly improved the software or our project. Lessons learned require group discussion and probably a change in team habits, process or strategy.
Lessons learned satisfy one the following criteria:
- Issue that is putting the project at greater risk than necessary
- Issue that is causing significant inefficiency
- Issue that is significantly lowering quality
What is Not a Lesson Learned
A lesson learned is not a minor annoyance, a tweak to an existing process, or something that can be resolved between team members in the everyday process of getting work done. Team members should bring these types of issues up at meetings, or work them among the team members involved.
A minor issue, (i.e. not a lessons learned), satisfies one of these criteria:
- Tweak to an existing process
- Minor annoyance or gripe
- Can be resolved by just picking up the phone, or discussing via email, or weekly meeting
- Does not require significant change in habits or processes
Things We Should Keep Doing
Things We Should Change
Do Better
- We need to make sure that all development is following our process as documented in the SMP and software development tickets in JIRA. We found significant issues during late stage development that would all be done if the checklists in those processes were used during development.
- We need to test the samples earlier and more frequently on all 3 platforms (perhaps weekly starting right before the Release process and continuing with the RCs). NOTE that the samples should be run on a clean build without (or before) running preparegmat, to find missing input data files.
- After editing sample scripts and before committing and pushing, run the entire relevant folder so that interdependent issues are captured. (Example: some scripts were deleted that were #Include scripts. If the related folder had been run, that would have been caught.)
- Some code is missing elements that should be added before being pushed to the main repository. Examples are the block at the top of the file identifying the copyright information, and comments of class parameters and methods. We need to do do better in reviewing code before pushing it because the coder is much less likely to make those changes later in the process.
- Improve regression testing process
- Mac tests need to be run regularly with results emailed out just like other platforms. We got lucky here but could have found issues late in release.
- Need to update regression test process to include API tests. API tests cannot be run in the same RunDef as the rest of GMAT regression tests because they require a different mode. We need to create a new master test runner file that sets up multiple run defs
Start Doing
- There are release processes that should be moved to development processes so they occur early and ensure nightly builds are as near release as possible. Requirements migration, RTTM mapping, wrap up tests, xml and rst docs.