Melete: Validating the Creativity Support Index as a Metric for Evaluating the Integration of AI In Software Pipelines

Murturi, Sokol ORCID: https://orcid.org/0000-0001-9466-8981, Walton-Rivers, Joseph ORCID: https://orcid.org/0000-0002-3406-0584, Scott, Michael ORCID: https://orcid.org/0000-0002-6803-1490, Yee-King, Matthew ORCID: https://orcid.org/0000-0001-6606-2448 and Gillies, Marco ORCID: https://orcid.org/0000-0002-3100-9230 (2025) Melete: Validating the Creativity Support Index as a Metric for Evaluating the Integration of AI In Software Pipelines. In: Proceedings of the Fourth International Conference on Hybrid Human-Artificial Intelligence. https://www.iospress.com/catalog/book-series/frontiers-in-artificial-intelligence-and-applications, 387unsure . IOS Press, Online. ISBN not yet known (In Press)

Preview

Text (CEUR_WS format) CEUR_WS_Melete__Validating_the_Creativity_Support_Index_as_a_Metric_for_Evaluating_the_Integration_of_AI_In_Software_Pipelines (1) (1).pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract / Summary

This paper evaluates the Creativity Support Index (CSI) for mixed-initiative artificial intelligence (MIAI) pipeline development. Determining the quality of the interaction between agents and users is valuable given the complex nature of MIAI pipelines. Within academic literature, the CSI is regarded as a practical measurement for assessing the usefulness of software. However, real-world validation of the CSI as a measurement tool for MIAI pipelines has not yet been established.

We compared two undergraduate cohorts with a sample size of 297, who completed a level-design task using 'Melete,' an MIAI pipeline, participants reported their experiences using CSI. We conducted a factor analysis to determine the validity of individual components of the CSI.

Analysis of the responses indicates the CSI is a valuable measurement tool for testing MIAI pipelines, with limitations. We make recommendations to improve the CSI including; the disentanglement of measures and the development of a robust set of questionnaires.

View Record (staff only)