Model cards - https://arxiv.org/abs/1810.03993
Are "short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups... and intersectional groups... that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information."
Model cards were motivated by systematic bias in commercial applications that were discovered only after the models were released. To counter that, the authors "advocate for measures of model performance that contain quantitative evaluation results to be broken down by individual cultural, demographic, or phenotypic groups, domain-relevant conditions, and intersectional analysis combining two (or more) groups and conditions." The emphasis on ethical aspects of the measurements is a distinguishing feature of model cards, compared to other proposals to document models.
Datasheets for Datasets - https://arxiv.org/abs/1803.09010
"Document [the dataset] motivation, composition, collection process, recommended uses, and so on. [They] have the potential to increase transparency and accountability within the machine learning community, mitigate unwanted biases in machine learning systems, facilitate greater reproducibility of machine learning results, and help researchers and practitioners select more appropriate datasets for their chosen tasks.''
The motivation behind the proposal was the electronics industry, where every component has a datasheet that describes its operating characteristics and recommended uses. In machine learning, data is the input for model training. Using the wrong dataset, or using a dataset outside of its original intent, or even not understanding well enough the limitations of a dataset, has dire consequences for the model. However, ``[d]espite the importance of data to machine learning, there is no standardized process for documenting machine learning datasets. To address this gap, we propose datasheets for datasets.''
This template allows easy creation of datasheets based on the work by Gebru, Timnit et al. “Datasheets for Datasets.” ArXiv abs/1803.09010 (2018): n. pag.
This template is inspired by the already proposed datasheet template by Gebru et al. (2018) and slightly adapted to serve two main purposes: dataset usage in Machine Translation (MT) and dataset consumer-oriented. By doing so, we are making a call to the community to work on these datasheets, independently of being the dataset author.
Marta R. Costa-jussà and Roger Creus and Oriol Domingo and Albert Domínguez and Miquel Escobar and Cayetana López and Marina Garcia and Margarita Geleta