This is the final installment in a three-part series featuring a case study in large scale culture transformation. Read Part One and Part Two now.

By Beverley Patwell, Donna Gray and Steve Kanellakos

Our Evaluation Framework
It is difficult to evaluate complex change. There are many OD models available to assess programs, but most address planned change. They fail to take into account emergent change. We designed the SE Leads Program Evaluation process to evaluate both.

Our model is built on the foundations of Donald Kirkpatrick’s Four Levels to evaluate learning and training programs (1993) and the elements of Classical and Postmodern OD as described by Gervase Bushe and Robert Marshak (2008). Since we were working on shifting culture, we especially wanted to reflect on and evaluate learning as it showed up in the emergent changes throughout the SE Leads program. We therefore built into our evaluation methodology numerous “snapshots” and “milestones” that would allow us to assess what everyone was learning at a given moment in time. Our Evaluation Frame­work was thus comprised of assessing three overarching areas:

• Vision, Context, Goals and Objectives – Our model took into account vision and context as well as goals and objectives. Many evaluations focus only on goals and objectives in the form of cost savings and outcomes, using quanti­tative data that all departments map into their business strategies. But this fails to examine progress towards the purpose—the overarching vision and context for the project. This requires qualitative data. For this reason, we periodically interviewed the SE Leads and asked them to describe in words how their departments and branches were going beyond a basic understand­ing of the meaning of SE to truly inte­grating the framework, mindset, and practices in their daily work. This evalu­ation helped us assess the strength of the real culture shift that was underway.
• Personal Learning and Insights—At the beginning of the individual SE proj­ects, many of the Leads automatically designed their projects using a classical planned change approach. They identi­fied a start and finish date and defined a fixed outcome they wanted to achieve. However, they soon realized that they had to adapt or even alter their plans due to emerging circumstances and events as they arose. Throughout the SE Lead Workshops and other leadership activities, they learned how to adjust their personal leadership approach and redesign their plans, often requiring the collaboration and inclusiveness of other stakeholders, groups, and connections with other departments to gain more impact. To measure their progress, we asked them to complete an open-ended written exercise called the SE Leads Reflections on Fostering a Culture of Service Excellence, where they had to comment on their personal learning journey. This self-reflection process proved invaluable in raising their con­fidence and boosting their insights into how to better lead and manage their SE projects.
• Relationships and Community—This element in our evaluation strategy was critical to the success of the entire project. Since one of our key objectives was to build more effective working relationships and a network of SE Leads across the City, we decided to assess the strength of their community and the connections between them. We did this using formal techniques such as evaluation surveys, but we also used informal techniques such as tapping into the feedback we received about how well the SE Leads bonded and shared knowledge during the group discussions they had in their learning clusters where they used mod­ules from CoachingOurselves, a man­agement and leadership development process created by Henry Mintzberg and Phil LeNir. In the end, we were pleasantly surprised to find that the SE Leads had formed a very close commu­nity, with 100% of them volunteering to continue their role in the coming years as mentors for the next group of SE Leads.

All three of these evaluation techniques were heavily weighted toward encourag­ing reflection and conducting strategic conversations with the SE Leads to assess the effectiveness and impact of their actions. The key influence on our evalua­tion framework was the work of Mintzberg, who emphasizes that what counts in the aftermath of a Learning and Develop­ment initiative is impact. In other words, does the learning of those who are trained transfer back into the organization beyond the manager participating in a session.

This philosophy inspired us to frequently seek to capture the impacts that would help us better gauge the effectiveness of our pro­cess and the inside story of culture change. The other influence comes from Beverley Patwell and Edith Whitfield Seashore who describe the influence of the choices lead­ers make in their Use of Self and the ripple impact they can have on teams and the organization.

Results of the SE Strategy
In the course of three years, from 2007 to the end of 2010, we witnessed a profound and quantifiable transformation in city ser­vices due to the implementation of SE. The list of individual corporate and departmen­tal SE accomplishments is extensive and would take pages to detail.

More importantly, culture change at the department level has been extraordi­nary. To date, the SE modifications speak to a wide variety of activities that have heightened employee engagement and the quality of decision making that impacts customers. We have seen more than 200 efficiency initiatives accomplished to date, many emphasizing new efforts at automa­tion. There are also new employee recogni­tion programs based on service excellence.

Of special note, silos are coming down; we are seeing far more joint meetings across city departments, and more effective and strategic working relationships that had not existed before. There are new com­munication efforts among departments to share client experiences. The client is now on the radar of every department. Every employee understands their “line of sight” and individual impact on the citizen’s expe­rience. And many departments are still in the midst of implementing follow-on plans that will make even more changes.

Our Key Insights about Culture Shift
Most people are fearful and resist organi­zational culture change. This case study explains the process and strategies that we took to understand the City of Ottawa’s culture and the pride that its employees have about working there. This foundation helped us, as the OD team, to understand what was sacred about the organization’s values and culture, but it also pointed out clearly what operations and attitudes had to change. If we had to list takeaways and key insights in our process, we would identify the following:

1. Think in Terms of Small Shifts—Rather than attempting to use brute force to implement a sudden transformation in the work styles and attitudes of city employees, we chose to frame the pro­cess as a series of small but steady cul­ture shifts. Combined, these eventually created momentum and synergy, leading to a profound and sustainable change in the entire culture.
2. Strong Focus on Leadership Develop­ment – Our team focused heavily on leadership development for an entire year before embarking on the specific elements of the change initiative. We created a strong framework and process to educate and orient the city’s senior executives, Branch General Managers, and Shared Service Directors around the concepts of SE. As leaders, they were responsible for introducing SE in their branches and departments and overseeing corporate and departmental SE plans. Obtaining their buy-in and commitment to change was a major factor in our success.
3. Strong Project Management—Our team adhered to a rigorous top-down project management philosophy by creating specific plans, timelines, and firm milestones to introduce SE to all stakeholders in the city. We insisted on adherence to honor corporate and departmental commitments to SE so as to keep the action moving forward. This strategic decision proved invaluable in overcoming resistance to change while maintaining focus, continuity and the momentum necessary for large-scale, complex systemic change.
4. Recognition of the Importance of Knowledge Transfer—We recognized that in large-scale transformations, it is vital to use the knowledge of the few to impact the many. Our strategic roadmap included many learning initia­tives—workshops, retreats, sympo­siums—aimed at relaying information about SE to every level of management and staff. Other initiatives were sup­ported with coaching and mentoring to develop a critical mass of leaders who could manage the SE implementation in their departments. The 50 SE Leads proved to be key resources and motiva­tors, helping department managers interpret and create new SE practices quickly and purposefully.
5. Focus on Building Community—Many elements in our strategic planning process were aimed at building a community around SE. We needed to encourage people to network and share challenges, knowledge, and success stories that would ultimately help them to complete the tasks required for their SE plans. One of our most significant victories in this regard was being able to finally break down silos among departments in the city and create a shared understanding of the teamwork required to implement SE across the city and across functions.
6. Commitment to “Use Work, Not Make Work”—An important element in the success and rapidity of the effort was our decision to “use work, not make work.” In all of our awareness forums and training workshops, we had lead­ers, managers, and SE Leads use their actual work experiences to identify successful SE experiences, gaps, and improvement strategies rather than working on a hypothetical or textbook situation. This kept their work real and meaningful.
7. Emphasis on Continuous Evaluation and Impact—We sought to rethink and develop new OD evaluation practices that would take into account not just planned but emergent changes and adaptations. One reason for this was our conviction that we could not force everyone in a large organization to accomplish change at the same pace. We had to create ongoing feedback mechanisms that allowed us to con­stantly measure results, keep track of themes and patterns that emerged, and make new recommendations as needed. A second reason was that we believed it was vital to constantly measure the quantifiable and qualita­tive impact that these initiatives were having on the culture.
8. Use of Internal and External OD Consultants—Because of the size of the organization, we recognized that we had to augment the original staff of 6 OD internal consultants with nearly 20 new consultants who were hired to assist with coaching and training managers and departmental staff. We trained these new consultants as part of the SE Leads program so they were learning simultaneously with the SE Leads and department manag­ers. This proved invaluable in that the OD consultants developed a strong shared sense of the challenges every­one faced, which helped them become accepted and respected advisors to the departments working alongside the SE Leads.
9. A Long-term Shared Vision—Right from the start of our effort, we rec­ognized that culture change requires a long-term commitment. We were patient, yet steadfast in our dedication to see the change through. In the first year, we focused on creating alignment and a common vision of SE with the Senior Leadership team. Following this process, we created a strong SE leader­ship development strategy and road­map for the coming years. This strategy included SE plans comprised of many initiatives to take place over three years. We directed our efforts at every level of management (executive, middle, and line supervisors) as well as employees. As stated earlier, we did not impose this vision in a fixed form on the many stakeholders involved. We asked each department to discover and interpret SE in their own context, shifting its practices relative to their own func­tions and at their own pace. In many ways, the effort was both top-down and bottom up, yet sharing the same overall long-term vision of a new culture.

We believe our process is transferable to any organization that needs to build employee engagement, and deeply under­stand their culture to achieve success. Ottawa is now benefitting from a highly engaged city government workforce that goes to great lengths to satisfy its citizens who utilize thousands of city services day after day. The culture shift is evident and palpable. In the world of OD, it would be hard to ask for a better result.

References
Bushe, G., & Marshak, R. (2008). Postmod­ern turn in OD. OD Practitioner 40(4), 9–11.
Heintzman, R., & Marson, B. (2005). Peo­ple, service and trust: Is there a public sector service value chain? International Review of Administrative Sciences, 71(4), 549–575.
Kirkpatrick, D. (1993). Evaluating training programs: The four levels (1st ed.). San Francisco, CA: Berrett-Koehler.
Mintzberg, H. (2011). From management development to organizational devel­opment with impact. OD Practitioner, 43(3), 25–29.
Patwell, B., & Seashore, E. (2006).Triple impact coaching: Use of self in the coach­ing process. Columbia, MD: Bingham House Books. 

Beverley Patwell, MA, CHRP, is President of Patwell Consulting Inc. She is an organizational development practitioner with over 25 years experience. In addition to her consulting practice she teaches in Mendoza College’s Executive Leadership Program at the University of Notre Dame and American University’s Masters in Organizational Development. She is an associate coach with the Niagara Institute and co-author with Edith Whitfield Seashore, of Triple Impact Coaching: Use of Self in the Coaching Process. She can be reached at bpatwell@patwellconsulting.com. 

Steve Kanellakos, MPA, is the Deputy City Manager of City Operations at the City of Ottawa. In this role, he is responsible for Community and Social Services, Emergency and Protective Services, Parks, Recreation and Cultural Services, Public Works, Public Health, Corporate Communications, Human Resources, Information Technology Services, and Organizational Development and Performance. He can be reached at Steve.kanellakos@ottawa.ca. 

Donna Gray, Director of Organizational Development and Performance Department at the City of Ottawa, has over 19 years of experience with the City of Ottawa and is responsible for organizational development, corpo­rate planning, performance management, corporate policy administration, strategic community and client initiatives, corporate project management, and the corporate implementation of the Accessibility for Ontarians with Disabilities Act. She oversees the delivery of services to citizens through the 311 Contact Centre and the seven Client Service Centres. She can be reached at Donna.gray@ottawa.ca.