Development of an
Agile Team Assessment
Carrie Schlauch & Ingrid Molan
INTRODUCTION
The Agile Team Assessment (ATA) measures team agility along with four factors: Trust, Clear Direction, Self-Organization, and Learning.
These factors are critical to creating a strong team foundation (Trust and Clear Direction) and facilitating adaptability in rapidly-changing environments (Self-organization and Leaning). The factors impact each other in a cyclical way, with Trust as the initial starting point from which the others build.
Assessing teams along these factors allows for targeted identification of a team’s strengths and areas of development, which is the basis for many team development processes.
ATA Version 1.0 collects responses to 63 positively-worded items using a 5-point Likert scale.
Data from 444 anonymous respondents from more than 50 separate teams to ATA Version 1.0 were analyzed to assess the theoretical model.
The analysis also had the further goal of ATA incremental improvement.
(Since 2019 the tool has been revised and currently a trial is in progress as a pre-/post- measurement of team-based intervention.)
Working for the greater good. Inspiring people, designing jobs and leading organizations
19th Congress of the European Association of Work and Organizational Psychology – Turin, 2019 May
THEORETICAL MODEL
TRUST
Create a psychologically safe environment with high trust, cohesion and openness.
CLEAR DIRECTION
Create clarity on the team’s purpose, objectives, deliverables, and roles.
SELF-ORGANIZATION
Empower the team to self-organize, establish effective team practices, self-monitor and adapt.
LEARNING
Exchange and respond to customer and peer feedback, learn from failure and strive for improvement.
KEY ANALYTIC RESULTS
– Internal consistency (α) of all factors > 0.70
– Mean inter-item correlations
Trust: 0.43
Clear Direction: 0.42
Self-Organization: 0.35
Learning: 0.37
– Principal components analyses (PCA): 4-factor
Variance accounted for: 47%
Trust (F1) and Clear Direction (F2) well-defined; >80% of associated items load 0.40+
Self-Organization (F3) and Learning (F4) are less well-defined; ≤50% of associated items load 0.40+
– Means of Trust and Clear Direction > Means of SelfOrganization and Learning (paired sample t-tests; p<0.01)
– Results of serial mediation (via PROCESS v3.3, 2019; Hayes, 2018) to test the theoretical model path (unstandardized coefficients shown):
OUTCOMES
There was modest support for the 4-factor model, with Trust and Clear Direction more clearly defined than SelfOrganization and Learning. An exploratory 3-factor PCA (Self-Organization and Learning combined in a single factor) was more parsimonious with slightly better fit.
Serial mediation analyses support the theoretical model’s directional relations (partial mediation). Mediation analysis using a 3-factor model (IV: Trust; Mediator: Clear Direction; Outcome: Self-Organization and Learning combined) also supported partial mediation (details available).
Removal of poorly performing items from ATA Version 1.0 (high means, low variance, weak theoretical basis) led to improved fit in both 4-factor and 3-factor PCAs. Revision of (primarily) Self-Organization and Learning items and creation of several new items resulted in ATA Version 2.0 (61 items).
Limitations in this work include lack of longitudinal data to test directional relations, lack of external measures for criterion validation and/or convergent/discriminant validation, and a lack of demographic information to investigate the possible influence on response patterns.
Theoretical Models
Dickinson, T. L., McIntyre, R. M., (1997). A conceptual framework for teamwork measurement.
In M. T. Brannick, E.,Salas, & C. W. Prince (Eds.) Team performance assessment and measurement: Theory, methods, and applications.
New York: Psychology Press.
Dingsøyr, T., & Dybå, T. (2012). Team effectiveness in software development: Human and Cooperative aspects in team effectiveness models and priorities for future development. Presented at 5th International Workshop on Co-operative and Human Aspects of Software Engineering (CHASE). Zurich, Switzerland, 2 June 2012. New York: IEEE.
Eddy, E.R., Tannenbaum, S. I., Mathieu, J. E. (2013). Helping teams to help themselves: Comparing two team-led debriefing methods. Personnel Psychology 66(4).
Lewis, M. W. & Smith, W. K. (2014). Paradox as a metatheoretical perspective. The Journal of Applied Behavioral Science, 50(2), pp. 127-149.
Lewis, M. W., Andriopoulos, C. & Smith, W. K. (2014). Paradoxical leadership to enable strategic ggility. California Management Review, 56(3), pp. 58-77.
Marks, M.A., Mathieu, J. E., & Zaccaro, S. J. (2001) A temporally based framework and taxonomy of team processes. The Academy of Management Review, 26(3), 356-376.
Wageman, R., Nune, D. A., Burruss, J. A., & Hackman, J. R. (2008). Senior leadership teams: What it takes to make them great. Harvard Business School Press: Boston
Statistical Analyses
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.
Clark, L. A. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.
DeVellis, R. F. (2003). Scale development: Theory and applications (2nd Edition). Thousand Oaks, CA: Sage Publications, Inc.
Hayes, A. F. (2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd edition). New York: The Guilford Press.
Hayes, A., F., (3 February 2019). PROCESS version 3.3 [SPSS macro]. Retrieved from: http://processmacro.org/index.html
Preacher, K. J., & Leonardelli, G. J. (2010-2019). The calculation for the Sobel test: An interactive calculation tool for mediation tests. Retrieved from: http://quantpsy.org/sobel/sobel.html
Piedmont R.L. (2014) Inter-item Correlations. In: Michalos A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht.