Improve More, Experiment Better

Imagine a scientist in a white lab coat, two beakers in hand. You ask what she’s doing, and she says, “I’m conducting an experiment!” When you ask what the experiment entails, she responds, “I don’t know! I’m going to combine these liquids and see what happens!”

Obviously no self-respecting scientist would call this an experiment. However people in almost any other line of business conduct these “experiments” all the time! “We’re going to build an open-concept work area and see what happens.” “We’re buying this new design tool. Let’s see what happens.” “We’re going to cancel our monthly staff meeting and see what happens.” Agility allows teams to pursue continuous improvement through experimentation, but only if we’re actually experimenting. For a real experiment, you need a real hypothesis—a best guess about what will happen. “See what happens” is not a helpful hypothesis because you can seldom disprove it. Something almost always happens! Keep on generating those ideas, but ask yourself these questions: What do I think is going to happen? It’s not an experiment if you don’t have a disprovable hypothesis (at the end, you must know whether your guess was right or not). When you implement this change, what will happen? Will productivity go up? Will the number of bugs discovered decrease? Be specific about the changes you expect. How and when will I measure my results? Some metrics are easy to track (e.g. number of bugs, call hold time, customer clicks, etc.), but others are very difficult (e.g. employee stress, quality improvement, customer engagement). Make sure you’re choosing metrics that actually prove or disprove your hypothesis, not just the easiest metrics to access. Who is involved in this experiment? Most of the changes we make impact other people. Choose enough participants so that you are learning from several perspectives, but not so many participants that—if your brilliant idea goes badly—the effects are catastrophic. “Narrow the blast radius,” as my coworker says. How long am I going to run this experiment? Change is hard, but it’s even harder when you don’t know how long it will last. Choose an appropriate timeline for your experiment and communicate it. This will keep you from monitoring a never-ending experiment, and will build trust with your participants. What other factors might confuse my results? If you’ve got a new design for your website and you want to measure how many more visitors you are receiving, think about other factors that might affect that result. Are you running your experiment on a weekend when web traffic is higher? Did you also put out a new web ad? Did you change your Google ad words? Limit those “confounding” variables when you can, and document them when you can’t. Keep in mind that your experiment is designed to be proven or disproven. If you thought your change was going to boost productivity and, instead, productivity plummeted—that’s a successful experiment! You and your team know more than you did before. You can make more informed decisions, and you never have to waste productivity through this change again. The only failed experiment is the one you don’t learn from.

Share this post

Leading SAFe

Take the Leading SAFe course to discover how companies can build business agility, and how to make SAFe work inside your organization. You’ll learn how SAFe helps you improve quality, productivity, employee engagement, and time-to-market. You’ll come away with an understanding of how to align your entire organization around the same clear objectives, and how to improve the flow of value and work from strategy to delivery. You’ll learn what makes companies more customer-centric and how to run key SAFe alignment and planning events, like PI planning.

Materials presented in this 2 day class are the latest from SAI and upon completion you will be registered for the SAFe SA Certification with Engaged Agility certification exam.

The 2-day course, exam, and 1 year of certification license is included in the price of the course.

All of our classes are “Confirmed to Run” so register with confidence!

Gabriel Pacheco

Gabriel Pacheco is an executive at a multinational company, working as a consultant in Digital Transformation, Organizational Agility, Management, and Leadership for large companies in Brazil and Latin America. With over 20 years of experience in technology and working with Lean/Agile since 2009, he stands out for his strategic vision, ability to lead organizational transformations and develop high-performing leaders and teams.

Gabriel has been a SAFe Practice Consultant (SPC) since 2020 and holds various certifications, including SA, PSM I, CSPO, KMP, KCP, PMP, among others. His passion for sharing knowledge has led him to train over 1000 people in public and private training sessions and speak at some of the largest agility events in Brazil.

In addition to his corporate career, Gabriel is also an entrepreneur and mentor in career growth, management, and leadership.  He is committed to helping people reach their maximum potential.

Confirmed to Run

At Engaged Agility, all of our classes are “Confirmed to Run” – even with just one student. We’re committed to providing high-quality education and personalized attention, no matter the class size. Smaller class sizes offer unique benefits, including more one-on-one time with our expert instructors and a collaborative, intimate learning environment.

You can expect a rich and rewarding educational experience that meets your needs, whether you’re enrolled in a class with one or many students. If you’re the only one enrolled, we’ll let you know before class and give you the option to take the class one-on-one or choose a future class with more students.

Join us for a tailored and flexible learning journey that helps you achieve your goals. Our “Confirmed to Run” classes provide peace of mind and a commitment to your success, every step of the way.