Development of an

Agile Team Assessment

Carrie Schlauch & Ingrid Molan

INTRODUCTION

The Agile Team Assessment (ATA) measures team agility along with four factors: Trust, Clear Direction, Self-Organization, and Learning.

These factors are critical to creating a strong team foundation (Trust and Clear Direction) and facilitating adaptability in rapidly-changing environments (Self-organization and Leaning). The factors impact each other in a cyclical way, with Trust as the initial starting point from which the others build.

Assessing teams along these factors allows for targeted identification of a team’s strengths and areas of development, which is the basis for many team development processes.

ATA Version 1.0 collects responses to 63 positively-worded items using a 5-point Likert scale.
Data from 444 anonymous respondents from more than 50 separate teams to ATA Version 1.0 were analyzed to assess the theoretical model.
The analysis also had the further goal of ATA incremental improvement.

(Since 2019 the tool has been revised and currently a trial is in progress as a pre-/post- measurement of team-based intervention.)

Working for the greater good. Inspiring people, designing jobs and leading organizations
19th Congress of the European Association of Work and Organizational Psychology – Turin, 2019 May

THEORETICAL MODEL

TRUST

Create a psychologically safe environment with high trust, cohesion and openness.

CLEAR DIRECTION

Create clarity on the team’s purpose, objectives, deliverables, and roles.

SELF-ORGANIZATION

Empower the team to self-organize, establish effective team practices, self-monitor and adapt.

LEARNING

Exchange and respond to customer and peer feedback, learn from failure and strive for improvement.

agile team assessment model

KEY ANALYTIC RESULTS

– Internal consistency (α) of all factors > 0.70

– Mean inter-item correlations

Trust: 0.43
Clear Direction: 0.42
Self-Organization: 0.35
Learning: 0.37

– Principal components analyses (PCA): 4-factor

Variance accounted for: 47%
Trust (F1) and Clear Direction (F2) well-defined; >80% of associated items load 0.40+
Self-Organization (F3) and Learning (F4) are less well-defined; ≤50% of associated items load 0.40+

– Means of Trust and Clear Direction > Means of SelfOrganization and Learning (paired sample t-tests; p<0.01)

– Results of serial mediation (via PROCESS v3.3, 2019; Hayes, 2018) to test the theoretical model path (unstandardized coefficients shown):

agile team assessment key results

OUTCOMES

There was modest support for the 4-factor model, with Trust and Clear Direction more clearly defined than SelfOrganization and Learning. An exploratory 3-factor PCA (Self-Organization and Learning combined in a single factor) was more parsimonious with slightly better fit.

Serial mediation analyses support the theoretical model’s directional relations (partial mediation). Mediation analysis using a 3-factor model (IV: Trust; Mediator: Clear Direction; Outcome: Self-Organization and Learning combined) also supported partial mediation (details available).

Removal of poorly performing items from ATA Version 1.0 (high means, low variance, weak theoretical basis) led to improved fit in both 4-factor and 3-factor PCAs. Revision of (primarily) Self-Organization and Learning items and creation of several new items resulted in ATA Version 2.0 (61 items).

Limitations in this work include lack of longitudinal data to test directional relations, lack of external measures for criterion validation and/or convergent/discriminant validation, and a lack of demographic information to investigate the possible influence on response patterns.

REFERENCES

Theoretical Models

Dickinson, T. L., McIntyre, R. M., (1997). A conceptual framework for teamwork measurement.
In M. T. Brannick, E.,Salas, & C. W. Prince (Eds.) Team performance assessment and measurement: Theory, methods, and applications.
New York: Psychology Press.

Dingsøyr, T., & Dybå, T. (2012). Team effectiveness in software development: Human and Cooperative aspects in team effectiveness models and priorities for future development. Presented at 5th International Workshop on Co-operative and Human Aspects of Software Engineering (CHASE). Zurich, Switzerland, 2 June 2012. New York: IEEE.

Eddy, E.R., Tannenbaum, S. I., Mathieu, J. E. (2013). Helping teams to help themselves: Comparing two team-led debriefing methods. Personnel Psychology 66(4).

Lewis, M. W. & Smith, W. K. (2014). Paradox as a metatheoretical perspective. The Journal of Applied Behavioral Science, 50(2), pp. 127-149.

Lewis, M. W., Andriopoulos, C. & Smith, W. K. (2014). Paradoxical leadership to enable strategic ggility. California Management Review, 56(3), pp. 58-77.

Marks, M.A., Mathieu, J. E., & Zaccaro, S. J. (2001) A temporally based framework and taxonomy of team processes. The Academy of Management Review, 26(3), 356-376.

Wageman, R., Nune, D. A., Burruss, J. A., & Hackman, J. R. (2008). Senior leadership teams: What it takes to make them great. Harvard Business School Press: Boston

Statistical Analyses

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.

Clark, L. A. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.

DeVellis, R. F. (2003). Scale development: Theory and applications (2nd Edition). Thousand Oaks, CA: Sage Publications, Inc.

Hayes, A. F. (2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd edition). New York: The Guilford Press.

Hayes, A., F., (3 February 2019). PROCESS version 3.3 [SPSS macro]. Retrieved from: http://processmacro.org/index.html

Preacher, K. J., & Leonardelli, G. J. (2010-2019). The calculation for the Sobel test: An interactive calculation tool for mediation tests. Retrieved from: http://quantpsy.org/sobel/sobel.html

Piedmont R.L. (2014) Inter-item Correlations. In: Michalos A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht.

FIND MORE IDEAS