ECSE 437: Software Delivery
McGill University, Montréal, Canada
Fall 2023
Learning Outcomes
Software development used to be a highly fragmented process. After developers have completed a set of enhancements and defect fixes, a candidate version of a release would be handed off to Software Quality Assurance (SQA) teams, who would check the candidate for defects. Next, the SQA team would hand the RC off to release engineering teams, who would prepare the software for release. After completing the official release process, software operators would install and configure the RC in production environments, where users can finally interact with it.
This fragmented approach to software engineering has largely been abandoned in industry for two reasons. First, since large quantities of enhancements and defect fixes would be grouped together into one release candidate, releases would encompass a large amount of change in the software system. These large releases were deemed risky both from a development organization perspective (i.e., field-reported defects can invalidate weeks or months of development effort) and a user adoption perspective (i.e., with so much change in the software system, it may be difficult to migrate existing installations to the new release). Second, the stakeholders on different teams would solely focus on tasks that fall within their purview, often leading to a lack of oversight of the whole development, release, and operations processes. Instead, modern development teams have embraced the DevOps mindset, i.e., a holistic view of the development, release, and operations processes. Enabling this mindset is a vast array of development, release, and operations tools, techniques, and practices.
Modern software delivery teams are tasked with selecting appropriate software delivery tools and assembling them to produce integration, release, and deployment pipelines that connect development, release, and operations processes. The material in this course will mainly be delivered through hands-on experience in developing integration, release, and deployment pipelines.
More specifically, students will:
- Be introduced to the challenges of modern software delivery
- Gain experience with several key tools (build and test automation, version control management)
- Design and develop integration, release, and deployment pipelines for a large application
The main objective of this course is to provide students with the skills that are needed to design, develop, and maintain the integration, release, and deployment pipelines that are at the centre of modern DevOps-based software releasing practices.
×
Get access to the official content in myCources at ECSE-437-001 or close it if it is in your way.
Instructor
Dr. Majid Babaei
Email address: majid.babaei@mcgill.ca,
Personal website:www.majidbabaei.com,
Office: 680 Sherbrooke Street West, 13th floor, room#1324
Department: School of Continuing Studies.
Online live Sessions, over Microsoft Teams, are held on Fridays 3:30pm (by appointment). These sessions, usually,
run for 30 minutes. However, they may also run for less than that in case there are no further questions or discussions.
Lecture slides will be posted over MyCourses. If you have questions and need clarifications about the lecture slides, then the live sessions will be the perfect location to get your questions answered and
clarified.
Teaching Assistants
Course Textbook
I will be using a series of textbooks to teach this course:
-
Achieving DevOps : a novel about delivering the best of Agile, DevOps, and microservices
-
Learning DevOps
-
CODE REVIEW GUIDE - OWASP Foundation
-
Implementing effective code reviews : how to build and maintain clean code
-
Software build systems : principles and experience
-
Static Program Analysis
-
Introduction to software testing
-
Continuous Delivery
Course Schedule
| Weeks
| Dates
| Topics
| Slides
| Textbook Sections
|
| 1
| Aug. 31
|
Course started in the 31st of August, 2023
Introduction, logistics, DevOps Culture, and Practices.
- Who is the instructor of this course?
- Who are the people in my team?
- Labs, Quizzes, Final Project, etc.
- Software Delivery in the past, today and future!
- Traditional software development is dead!
- Three axes of the DevOps movement
|
|
Learning DevOps: Sections 1 and 2
|
| 2
| Sept. 5, 7
|
Lab1 [due Sept. 19th]
Project Planning
- Azure Boards
- AZURE DevOps Organizations
- AZURE DevOps Projects
- Type of projects (Basic/Agile/Scrum)
- WrokItems
- Backlogs
- Sprints
************
Git Basics 1
- Centralized vs. distributed VCS
- What is Git ?
- Git vocabulary
- Git Repository Structure
- Creating snapshot
- Basic Git workflow
|
|
GitLab Cookbook: Chapters: 1, 2
|
| 3
| Sept. 12, 14
|
Git Basics 2
- How to skip the Staging Area
- Removing a file both from WD and SA
- How to ignore files in git?
- How to interpret the output of git diff
- Viewing history
- Unstaging files
- Restoring a file to an earlier version
************
Git Basics 3
- What is Branching?
- Create, show and switch to a branch
- Comparing branches
- Stashing
- Merging
- Fast-Forward Mergers
- 3-way Mergers
- Merge conflicts
- Aborting a merge
|
|
GitLab Cookbook: Chapters: 1, 2
|
| 4
| Sept. 19, 21
|
Lab2 [due Oct. 5th]
Git Advanced Topics
- Rebasing
- Cherry Picking
- Picking a file from another branch
- Collaboration on a project
- Centralized Workflow
- Integration Manager Workflow
Presentation Session 1 (2 TIME SLOTS)
[?] The Promises and Perils of Mining Git [link]
[?] Can Git Repository Visualization Support Educators? [ link]
************
Quiz 1 [Sept. 19th]
|
|
Code Review Guide: Chapters: 1, 2, and 3
Gerrit Code Review: Chapter: 1
|
| 5
| Sept. 20, 22
|
Code Review Intro
- Poor quality reviews
- Great Reviews
- Best practices for code review
- Some Examples
************
Gerrit Basics
- What is Gerrit?
- Build stability
- Gerrit Concepts
- Push for Code Review (3 cases)
- Review and vote
|
|
Code Review Guide: Chapters: 1, 2, and 3
Gerrit Code Review: Chapter: 1
|
| 6
| Oct. 03, 05
|
Lab3 [due Oct. 25th at 9:00AM]
Gerrit advanced topics
- Standard workflow
- Working with Change Series
- Conflict Resolution in Gerrit
- Working with Topics
- Working with Stable Branches
************
Presentation Session 2 (3 TIME SLOTS)
[?] Modern code review: a case study at google [link]
[?] Who Should Review My Code? [link1, link2]
[?] Characteristics of Useful Code Reviews. [link]
|
|
Software Build Systems: Chapters: 1, 2, and 3
|
| ---
| Oct. 10
| Fall Reading Break, No Class
|
|
|
| 7
| Oct. 14
|
Quiz 2
|
|
|
| 8
| Oct. 18, 20
|
Static Code Analysis
- What is it? Why do we need that? and how does it work?
- Manual Review
- Automated tools for performing static code analysis
- Examples (Buffer Overruns, Uninitialized variables, Memory Leaks, etc.)
- Features of advanced static analysis tools
************
Presentation Session 2 (2 TIME SLOTS)
[Taken] What is a docker container and its architecture? Where does it apply in DevOps? [A quick demo to show how it is used in a small project]
[Taken] What are Docker Based Build Systems? Compare them with traditional build systems. [A quick demo to show how they can be applied in a small project]
|
|
Static Program Analysis: Chapter: 1
Advanced Static Analysis for C++
|
| 9
| Oct. 25, 27
|
Course Project [due TBA at 9:00AM]
Software Testing
- What could happen when software goes bad?
- Goals of Testing Software?
- Model-Driven Test Design (MDTD)
- Test Desing, Automation, Execution, Evaluation
- High overview of a testing process
************
Software Testing
- Software testability
- Test case
- Executable test script
- A test automation framework
- JUNIT
- CPPUNIT
Presentation Session 2.5 (1 TIME SLOT)
[Taken] How to test Docker Images in the cloud? How it can be integrated it into a DevOps project? [with a short demo]
|
|
Introduction to Software Testing: Parts: 1, 2, 3, and 4
|
| 10
| Nov. 1, 3
|
Software Testing
- Why we should put testing first?
- Cost-of-Change curve
- agile software development methods
- System Tests in Agile Methods
- Coverage criteria
- Advantages of using coverage criteria
************
Presentation Session 3 (2 TIME SLOTS)
[Taken] What are the key challenges of testing distributed systems and how can we deal with these challenges?
[Taken] The Comparison of the best Agile Testing Tools: Worksoft, PractiTest, JunoOne, QMetry, Testiny, JIRA, Selenium WebDriver
|
|
Software Build Systems: Chapters: 1, 2, and 3
|
| 11
| Nov. 08, 10
|
Quiz 3
************
Continuous Integration / Continuous Delivery
- Reduce Risks
- Reduce Repetitive Processes
- Establish Greater Product Confidence
- What Prevents Teams from Using CI?
- When and How Should a Project Implement CI?
- How long does CI take to be set up?
- How to establish good day-to-day software development habits?
- Commit Code Frequently
- Do not Commit Broken Code
- Fix Broken Builds Immediately
- Write Automated Developer Tests
- Make Sure All Tests and Inspections Passed
|
|
Continuous Delivery: Chapters: 1, 2, and 3
|
| 12
| Nov. 15, 17
|
Continuous Integration / Continuous Delivery
- An example of CI
- Agile runs on CI
- How to use a CI tool
- Creating a self-triggered build
- Automate the packaging
- Automating the deployments
- Automating the testing
- Automating the code analysis
- Automating the deployments
************
Continuous Integration / Continuous Delivery
- What is Jenkins Pipeline?
- One continuous delivery scenario
- Declarative vs. scripted pipelines
- Scripted pipeline drawbacks
- Declarative Pipeline fundamentals
- Pipeline example
Presentation Session 3.5 (2 TIME SLOT)
[Taken] How does Kubernetes work? [with a short demo]
[Taken] How does Jenkins work? [with a short demo]
|
| textbook section
Learning Continuous Integration with Jenkins Chapters: 4, 5, and 6
|
Labs
Labs general directions:TBA
Lab Instructions and Files:
-
Lab 1 (Lab_1.pdf) [due Sep 20th]->extended to Sep 27thIn this lab, there are 12 (small) exercises. If you have never worked with any version control systems like Git the main challenge for you will be to understand the concepts, create an account and worke with the cloned repository. Your main task in this lab is to understand what kinds of benefits you will get by using a version control system rather than keep naming your file like: final.cpp, final2.cpp, reallyFinal.cpp, finalyReallyFinal.cpp, OMGfinal.cpp, etc. It is never ending! You never know which one is the actual final cpp file!
-
Lab 2 (Lab_2.pdf) [due Oct 04th] In this lab, there is only three exercises. During this lab, you will be using Gerrit, GitHub and GerritHub to add a reviewer, make a change and ask the reviewer manually/automatically to check your changes. Then you will get a notification and try to address the feedback from the reviewer. In one part of this lab you should add TA and professor as the reviewer of your code! make sure you do that because if we don't get the message, you don't get the full mark!
-
Lab 3 (Lab_3.pdf, Lab_3.zip) [due Oct 29th]
This lab is challenging because it is your first time writing Makefile. In the first part of this lab, we will be working on a simple compilation process with just one source file. We will learn three main elements of a Makefile and how we can use them towards our goal which is (usually) generating an executable file.
In the second part of this lab, we will consider a bit more complex scenario, where the source code is broken down into multiple files. It is more challenging because now we should first compile those files to object code, an intermediate compilation stage. Next we should compile multiple object files to assemble an executable file.
Quizzes
-
Quiz 1 [on September 20th]This quiz covers lectures from week 1 to week 3, i.e., DevOps Practices and Git.
-
Quiz 2 [on October 14th] This quiz covers lectures from week 4 to week 6, i.e., Code Review, Gerrit and build systems.
-
Quiz 3 [on November 8th] This quiz covers lectures from week 7 to week 10, i.e., Static Analysis and Testing.
Final Project
Students will work on their final project in the same teams that they have worked on labs assignment. The task of each group is to use the concepts and tools that they have learned through the course to design, develop, test, and deploy a software project and provide a demo of their project. The following points need to be addressed in the final presentation of your project.
-
The software has to have at least seven components (or classes) that are coded into different files.
-
To manage different version of the software a version control systems like git must be used
-
You should demonstrate that you code has multiple branches, e.g., main, bugfix/…,feature/…, etc. that are merged into your main line of development every couple of days.
-
One student mush act as a code reviewer and send his/her feedback to the other student using code review system like Gerrit or GerritHub
-
Each team must demonstrate how unitary and integration testing performed to ensure the software is free of bugs in each release.
-
Each team must use an automatic build system.
-
Each team must use a static code analysis approach in their project.
-
Each team must use a continuous integration orchestration tool like Jenkins to build artifacts throughout the pipeline.
Note: You are free to choose any programming languages that you are more familiar with.
Late Submission Policy
-
If you submit your work 15 minutes after the deadline, then you lose 5% of the total grade.
-
If you submit your work 24 hours after the deadline, then you lose 10% of the total grade.
-
If you submit your work 48 hours after the deadline, then you lose 20% of the total grade.
-
If you submit your work 72 hours after the deadline, then your submission will not be considered.
Grade distribution
The grade breakdown is as follows:
- Lab 1 weights 6.0%
- Lab 2 weights 6.0%
- Lab 3 weights 6.0%
- Lab 4 weights 6.0%
- Lab 5 weights 6.0%
- Quiz 1 weights 10.0%
- Quiz 2 weights 10.0%
- Quiz 3 weights 10.0%
- Final Project weights 10.0%
- Final Exam weights 30.0%
- Bonus point 10.0%
You can earn the Bonus point in two ways:
- If you give a presentation (up to 5.0%)
- If in your final project you add extra steps and making significant contributions in the labs (up to 5.0%)