Data-driven software delivery process design
October 16, 2022
At any point in time, most businesses undergo some process redesign. Of the 128 organizations surveyed by Signavio for the report “The State of Business Process Management 2020”, 91% were undertaking at least one business process improvement project in 2019. Major business process redesign projects were the focus of about 40% of the surveyed organizations, and 36% were working simultaneously on ten or more process improvement projects.
Business process design can pursue different goals. Processes can be designed and redesigned to make them more cost-effective, improve products, or ensure that the processes align with the work culture and business goals. Process design can be applied to improve any business process related to customer support, human resources, or IT.
That said, although process design can help businesses improve their efficiency and productivity, it needs to be executed properly. But how exactly should these projects be implemented?
The answer to that question is as follows: business process design needs to use data heavily. In this article, we are going to introduce process design and the role of data in its success. When talking about process design, we will be putting a strong emphasis on IT – specifically, software delivery.
What is business process design?
“If I had one hour to save the world, I would spend fifty-five minutes defining the problem and only five minutes finding the solution."
- Albert Einstein, Physicist
Business process design is the creation of a business process or a workflow from the ground up. It involves analyzing an existing process, identifying areas that need change, and developing a new process that addresses the issues discovered in the legacy process.
Business process design is related to business process reengineering, business process improvement, and business process modeling. However, don’t confuse these approaches with each other. Although similar, they are distinct and have different goals.
- Business process design (BPD) aims to design a new process from the ground up (Mendling and Simon, 2006, September).
- Business process improvement (BPI) aims to make minor, incremental changes to existing processes (Bisson, et al., 2000).
- Business process reengineering (BPR) aims to make major changes to existing processes (Bevilacqua and Giacchetta, 2009).
- Business process modeling (BPM) aims to graphically represent an existing business process and its sub-processes (Rub and Issa, 2012).
- Workflow Management Systems (WfMS) encompass a greater level of granularity associated with BPM (Heinl, et al., 1999).
- Business Process Model and Notation (BPMN) is stewarded by the Object Management Group® (OMG®). BPMN is a standard that comprises a graphical representation notation specification for business processes in a business process model. BPMN has been propagated as ISO/IEC 19510 - Information Technology — Object Management Group Business Process Model and Notation (Nuzulita, et al., 2020).
One thing to remember here is that the team can view business process design as part of business process reengineering or even the exact same thing. BPR can include not only the restructuring or elimination of inefficient processes but also the creation of new ones whenever needed. With that in mind, the boundaries between business process design and reengineering can be blurry, and the two concepts can be synonymous in some situations.
Process design as part of continuous process transformation
“Without continual growth and progress, such words as improvement, achievement, and success have no meaning."
- Benjamin Franklin, Politician & Scientist
The approaches we listed above could be used individually, but a team could also use them to optimize business processes. Consider an example where a team wants to analyze and improve a waterfall software delivery workflow. The waterfall model is very inefficient for software development and is a good candidate for change (Thummadi and Lyytinen, 2020). Here’s how the team would go about this task at a very high level:
- To better understand how software delivery workflow works, the team would first apply business process modeling to it. With BPM, the team can get a visual insight into the software delivery process and identify redundant steps or weak links. For example, as a result of this step, the team could discover that the waterfall model’s sequential nature limits developers' ability to respond quickly to changes in customer needs.
- Based on BPM results, the team would then determine the extent of the optimizations that should be made to the software delivery process. If minor changes will suffice, the team could go with BPI. Otherwise, the team would most likely need to choose BPR.
- While analyzing opportunities for improvement via BPR, the team could decide to restructure some processes, eliminate others, and develop several new processes. Suppose we stick to our waterfall example. At this step, the team might decide to completely ditch the old waterfall process and switch to a more flexible continuous integration/continuous delivery (CI/CD) model.
- After BPR is complete, the team would keep an eye on the performance of the new process and apply BPI to it whenever issues are detected. For example, the team might discover that the manual approval of deployment in continuous delivery unnecessarily delays the delivery of feature updates to production. The team could switch to continuous deployment with automated deploys to production to fix this issue.
Our example shows how the four approaches can be organically mixed together to achieve the desired outcome. Don’t use the four methodologies in isolation – instead, combine them so the team can continuously improve workflows.
Why do software firms need process design?
“If you do not change direction, you may end up where you are heading.”
- Lao Tzu, Philosopher
If a business wants to optimize its workflows, why would it need to design them from scratch? Wouldn’t the team be able to achieve good results by just making tweaks to existing processes?
The answer depends on how much change a process needs to meet its KPIs (Stašák and Schmidt, 2018). In some cases, a small change might indeed be all that’s necessary to bring a process to perfection. However, processes that are really old, complex, or inefficient might need to be rebuilt from scratch.
When designing a process from the ground up, the team gets the chance to fine-tune its most minor details to achieve the best outcome. Besides, with process design, the team won’t need to think about how to integrate improvements into a barely working old process. Instead, the team can start from a new page and build a strong workflow.
At a high level, here are the primary benefits that the team gets by designing processes from the ground up:
- Measurability. First up, the team can build the new process with performance tracking in mind. By integrating analytics into the process, the team can track metrics more accurately, like lead times for changes and change failure rates.
- Standardization. The team can easily standardize the new process to align with the organization’s internal culture, policies, values, and business goals. Not only that, but standardization enables scalability and automation, which we’ll cover in the following points.
- Automation. If a process is standardized, its components that perform related functions are very similar. This situation can be extremely beneficial for automating the software delivery process. Let’s consider QA (quality assurance) as an example – if the code is standardized, QA engineers can easily develop automated routines that can test API endpoints, security, and performance across different areas of a project or multiple projects.
- Scalability. Scalability is another trait the team can ingrain in the new process from the beginning. If the process is standardized, the team can easily use its components as building blocks to expand its capabilities as the needs grow. Aside from that, the team can build the process around scalable software delivery tools from the very beginning to better meet the long-term performance and growth goals.
However, those benefits are not the only ones. What about the end goal of general business process design and process transformation (Mathrani, 2010)?
Every business will have its reasons for doing business process design. For reference, according to the “The State of Business Process Management 2020” report by Signavio, the reasons for business process change for the surveyed organizations in 2019 were as follows:
- Cost reduction and/or improved productivity (69%).
- Improvements in customer satisfaction (38%),
- Improvements in products or the creation of new products or lines of business (35%).
- Improvements in IT resource management (31%).
- Reduction of cultural resistance to process changes (24%).
- Government or business risk management (20%).
Business process design doesn’t have to focus on just one outcome. If the team does everything right, multiple goals can be achieved with business process design. However, hitting multiple goals is nearly impossible without data. Here’s where the concept of data-driven business process design comes into play.
What is the “data” component in business process design?
We now understand what process design is, but what about the “data-driven” component we discussed earlier? What is data, and why do we need it?
As far as business process design is concerned, data is information that allows us to:
- Better understand our current processes, their strengths, and weaknesses.
- Understand if the process meets our business requirements, customer needs, and relevant regulations.
- Identify improvement opportunities in existing processes.
- Design new processes that address the discovered issues.
The practice of using data in business process design isn’t original. For example, organizations have used the statistics-based Six Sigma method with varying degrees of success for decades. Prominent past practitioners of Six Sigma include Amazon, Boeing, and the United States Marine Corps. However, classic approaches like Six Sigma don’t work well in today’s fast-paced competitive markets.
Professional services company Deloitte points out that classical BPR methodologies like Six Sigma, Kaizen, and Theory of Constraints don’t consider the dimensions of people, processes, and technology when dissecting business processes (Al-Shammari, 2009). These methodologies don’t consider the bigger picture and have a very isolated view of business process design, optimization, and reorganization. They are too focused on the core process and don’t pay enough attention to external factors.
With modification, classical BPR methodologies can become usable even today. However, this is beyond the scope of this article – our goal now is only to demonstrate why data is important in business process design.
At a high level, here are the different dimensions of data that must be considered in business process design:
- Changing regulatory requirements. Regulations can change quickly, especially in areas that have to deal with data privacy and confidentiality. Industries that handle sensitive data – like finance or healthcare – should pay special attention to regulations because failure to comply with them can undermine the validity of a business process. If we are talking about software delivery, the team should ingrain compliance in every step, from handling customer data to code security. Among other things, teams should securely store personally identifiable information and ensure that their products comply with regulations like GDPR (General Data Protection Regulation) or COPPA (Children’s Online Privacy Protection Act).
- Changing customer preferences. The key strength of modern digital businesses is customer-centrism. Although old, classical businesses also realize the importance of customer preferences, they are still miles behind digitally native firms in this area. Paying careful attention to customer feedback is extremely important because it ensures that the team can quickly adapt the software products to changing customer preferences. If the team does everything right, it will be able to address the customers' issues faster than competitors do.
- People. People are the individuals responsible and accountable for a business process. The “people” dimension for software delivery could include software engineers, software architects, QA engineers, designers, and product managers. This dimension ensures that responsibility for the process is clearly outlined and properly distributed between employees.
- Technology. Technology encompasses the tools and software that support a process and simplify its automation and monitoring. Using the right toolsets is crucial for the efficiency and success of a software delivery pipeline. Technology tools the team can use to support the software delivery functions include CI/CD software, correctly configured computing infrastructure, and the programming frameworks that teams use to develop and maintain software.
- Processes. Processes are the activities that support the business’s functions. The team should not forget other workflows when designing a business process. The team needs to align the new process with existing ones to ensure that all the functions operate with one purpose – to meet the business goals.
When designing a business process, remember one thing – the team should not view the process in isolation from its environment. If the team neglects the data dimensions we’ve listed above, the team might inadvertently sub-optimize the process for some local metric.
For example, if the team decides to focus on bugs, it might spend way more time and resources on QA than it needs to. Eliminating minor bugs can improve code quality, but because very small bugs often don’t affect the user experience much, customers might not notice the effort. Consequently, the team will have wasted time on something that, in the end, didn’t deliver value to the business and the customer.
This outcome is precisely why data in modern business process design has so many angles and dimensions. We cannot view a business process through the lens of the process itself – we also need to consider other relevant factors inside and outside of it.
The four stages of software delivery process design
Business process design is a long, multi-stage process – for most businesses, it would include the following four stages:
“Eventually everything connects - people, ideas, objects. The quality of the connections is the key to quality per se.”
- Charles Eames, Designer
Let’s look at these stages below.
Analyzing current workflows is a critical step in business process design. With concrete real-world findings, the team can pinpoint exactly where it is experiencing issues today and can identify improvement opportunities. Business process modeling can be a beneficial tool at this step.
Throughout the analysis stage, software delivery teams can pay attention to things like:
- Redundant steps.
- Communication gaps.
- Poor coding practices.
- Poor response time to issues.
- Unacceptably high rates of bugs in code.
It’s also essential to conduct interviews with relevant stakeholders, such as IT managers, employees, and customers. Stakeholders can provide unique insight and help the team identify and treat high-priority issues first.
2. Process Outline
Based on the results of the analysis stage, the team can start building the perfect process. At a very high level, a business process should contain the following components:
- Inputs. Each business process needs to have one or more inputs that will initiate it. An example of a software delivery process input is a backlog of tasks. The process of coding a new feature or a fix starts from the need for change in the target application. When a new feature is needed, it’s added to the developers’ backlogs, so they know what to do next.
- Outputs. Business processes must have one or more outputs. Output is the result of the process’s operation. The deployment of new code to production is an example of a software delivery process output.
- Resources. Resources allow a business process to produce the desired output. For a software delivery process, resources can include employees (developers, QA engineers, etc.), code, CI/CD platforms (like GitLab or GitHub Actions), programming frameworks (like Node.js or .NET), and collaboration tools (like Microsoft Teams or Slack).
- Procedures. Procedures are the steps and activities that enable a business process to produce output. Procedures could include code development, quality assurance (including unit testing, integration testing, and security testing), and the deployment routine for a software delivery process.
- Responsible parties. These are the parties that are responsible for the efficient and successful operation of a process. Parties responsible for a software delivery process can include the product manager, developers, DevOps engineers, and front-end developers.
3. Business Process Modeling
At the stage of business process modeling, we bring our process outline into a graphical form. BPM allows the team to get a more visual understanding of the process. Additionally, flowcharts are an excellent way of showing the new business process to stakeholders.
Teams can visualize their new business process on a whiteboard, or they can use specialized BPM software. Physical media like paper or whiteboards can be used for initial sketching, while digital software can be used for sharing flowcharts and testing the performance of the process.
Speaking of performance testing, it is perhaps the most critical function of business process modeling. It allows teams to test their assumptions about the process and get practical insights into the performance of the new process.
Simulation can be a potent tool at the stage of business process modeling. Once a software delivery process is visualized, it can be moved to simulation software for performance and bottleneck testing. Not only that, but teams can quickly test different tweaks to their process to see how they affect it. As a result, teams can quickly evaluate dozens of different process configurations and choose the best one.
4. Preparation for Deployment
After a process is modeled, it needs to be prepared for deployment. The first thing to do at this step is to verify and validate the process.
- Verification aims to check that the process meets its technical requirements. For software delivery, verification can include checking if the delivery process has the expected architecture and conforms to technical specifications. Verification does not require the team to run the new process.
- Validation, on the other hand, is about ensuring that the process meets the business needs. To validate a software delivery process, the team would need to run it – be it in the real world or simulation software – and check if it matches the expectations regarding performance and efficiency.
After verification and validation, employees that will be engaged in the process need to be retrained and briefed about their new procedures and responsibilities. Employee education should not be limited to the specific tech stack that will be used in the process – it should also incorporate policies and the work culture that the team will adopt to make the new process sustainable. Software delivery teams can follow DevOps practices to ensure they have the right mindset for today’s competitive market.
Business process design isn’t the end of the road in business process optimization. After the new process is designed and deployed, a new phase starts – the phase of monitoring and continuous improvement (Mincheva and Stefanov, 2018).
Regulations and customer preferences can change daily, and the business strategy needs to reflect that. Business agility and desire to grow are critical here. No matter the current positive state of the processes, they might become a bottleneck for the business operations as the market changes.
As this happens, the team will discover weaknesses in the processes. That’s exactly when the team would reevaluate where it is today so that incremental changes can be made to ensure that the team is ahead or at least on par with the competition. Has the team considered some inexpensive seminars or courses to help familiarize newcomers to this topic:
- Business Process Design (BPD)
- Business Process Improvement (BPI)
- Business Process Reengineering (BPR)
- Business Process Modeling (BPM)
Bevilacqua, M., Ciarapica, F. E., & Giacchetta, G. (2009). Business process reengineering of a supply chain and a traceability system: A case study. Journal of Food Engineering, 93(1), 13-22.
Bisson, B., Folk, V., & Smith, M. (2000). Case study: how to do a business process improvement. The Journal for Quality and Participation, 23(1), 58-63.
Heinl, P., Horn, S., Jablonski, S., Neeb, J., Stein, K., & Teschke, M. (1999). A comprehensive approach to flexibility in workflow management systems. ACM SIGSOFT Software Engineering Notes, 24(2), 79-88.
Mathrani, S. (2010). A transformational model to understand the impact of enterprise systems for business benefits. (Doctoral Dissertation). Massey University, New Zealand
Mendling, J., & Simon, C. (2006, September). Business process design by view integration. In International conference on business process management (pp. 55-64). Springer.
Mincheva, A., & Stefanov, G. (2018). Business Process Model Optimization as Important Step of Business Evolution. In Proceedings of International Conference on Application of Information and Communication Technology and Statistics in Economy and Education (ICAICTSEE) (pp. 296-304).
Nuzulita, N., Djohan, R. S. A., & Roiqoh, S. (2020). Supply Chain Management Analysis Using the Business Process Model and Notation in the Midst of Covid-19 Pandemic. Journal of Accounting and Strategic Finance, 3(2), 185-198.
Rub, F. A. A., & Issa, A. A. (2012). A business process modeling‐based approach to investigate complex processes: Software development case study. Business Process Management Journal, 18(1), 122-137.
Stašák, J., & Schmidt, P. (2018). Key performance indicators versus business process metrics. International Journal of Advanced Operations Management (IJAOM), 10(2), 130-154.
Thummadi, B. V., & Lyytinen, K. (2020). How much method-in-use matters? A case study of agile and waterfall software projects and their design routine variation. Journal of the Association for Information Systems, 21(4), 864-900.
Tsolakis, N., Bechtsis, D., & Srai, J. S. (2018). Intelligent autonomous vehicles in digital supply chains: From conceptualisation, to simulation modelling, to real-world operations. Business Process Management Journal, 25(3), 414-437.