Periodic Reporting for period 2 - PROVENANCE (Providing Verification Assistance for New Content)
Berichtszeitraum: 2020-06-01 bis 2022-05-31
The online circulation of false and misleading information is a major threat to social and political stability. It is implicated in the resurgence of vaccine-preventable diseases, efforts to undermine the political process, and the amplification of social divisions. Yet, online disinformation is a complex problem with many contributing factors. In line with the recommendations of the High Level Expert Group on Online Disinformation (2018), PROVENANCE aims to empower citizens by making the context of online content more transparent and by promoting media and information literacy. The overall idea is to intervene in the ‘attention economy’ of digital media by equipping audiences with media and information literacy skills and by creating a pause before people hit "like" or "share", PROVENANCE hopes to slow down the spread of disinformation. This goal is supported by research which finds that judgement can be improved by asking people to think about content accuracy and by warning people about manipulation tactics.
To achieve this, PROVENANCE uses blockchain to record multimedia content that is registered by the PROVENANCE Social Network Monitor. The PROVENANCE Verification Layer then applies advanced tools for multimedia analytics (semantic uplift, image forensics, cascade analysis) to record any modifications to content and to identify similar pieces of content. A personalised Digital Companion caters to the information needs of end-users to help them navigate content and develop digital literacy competencies. In part, this is achieved through an iconographic Verification Indicator, which is accessed as a browser plugin. It contextualises individual pieces of content with relevant information including: the date and origin of the visual or textual content, its similarity to other content, the quality of the writing including the use of loaded or highly emotional language, and the degree of visual manipulation.
PROVENANCE is co-created with diverse representatives of civil society across three distinct use-cases: citizen information seekers, media literacy practitioners, and content creators. However, the findings will be applicable to any area in which social media and verification are important. The scientific and pragmatic insights gained through PROVENANCE will significantly advance the state of the art in intermediary-free solutions for content verification, understanding of information cascades and information sources on social media, the openness of algorithms, and user control over personal data.
To achieve this, PROVENANCE uses blockchain to record multimedia content that is registered by the PROVENANCE Social Network Monitor. The PROVENANCE Verification Layer then applies advanced tools for multimedia analytics (semantic uplift, image forensics, cascade analysis) to record any modifications to content and to identify similar pieces of content. A personalised Digital Companion caters to the information needs of end-users to help them navigate content and develop digital literacy competencies. In part, this is achieved through an iconographic Verification Indicator, which is accessed as a browser plugin. It contextualises individual pieces of content with relevant information including: the date and origin of the visual or textual content, its similarity to other content, the quality of the writing including the use of loaded or highly emotional language, and the degree of visual manipulation.
PROVENANCE is co-created with diverse representatives of civil society across three distinct use-cases: citizen information seekers, media literacy practitioners, and content creators. However, the findings will be applicable to any area in which social media and verification are important. The scientific and pragmatic insights gained through PROVENANCE will significantly advance the state of the art in intermediary-free solutions for content verification, understanding of information cascades and information sources on social media, the openness of algorithms, and user control over personal data.
As online disinformation is a complex problem, PROVENANCE has undertaken extensive testing with citizens and media literacy practitioners in Ireland and Spain to define the use case requirements and to develop a baseline of knowledge for evaluating the effectiveness of the PROVENANCE solution. To date, some 350 people have participated in pre-pilot and pilot studies. In parallel with this, PROVENANCE has developed the principal components of the system architecture, which includes a blockchain Verification Layer for multimedia content; an architecture for full blockchain integration; four fully integrated tools for textual and visual content verification; new techniques for monitoring information cascades; a Verification Layer and browser plugin; and a Personalised Companion.
To develop the blockchain verification layer, there is an on-going focus on the definition and application of a fingerprinting or asset registration process within the platform. Substantial work has been undertaken on the development of the tools for textual and visual content verification. An interactive demo of the Verification Indicator has been developed and there is ongoing testing with citizens and experts to ensure the overall design and the information provided are fit for purpose. In particular, as PROVENANCE aims to improve media and information literacy, partners are currently running workshops and evaluations with media and information literacy practitioners to (i) ascertain feedback for the refinement of the Verification Indicator and (ii) identify how it may be used by a range of practitioners (e.g. libraries, NGOs) to support the services they provide to citizens. The minimal user model has been defined for the Personalised Companion. In line with the above, the project has pursued a communication and dissemination plan that supports awareness among and engagement with key stakeholders as well as a revised exploitation plan.
To develop the blockchain verification layer, there is an on-going focus on the definition and application of a fingerprinting or asset registration process within the platform. Substantial work has been undertaken on the development of the tools for textual and visual content verification. An interactive demo of the Verification Indicator has been developed and there is ongoing testing with citizens and experts to ensure the overall design and the information provided are fit for purpose. In particular, as PROVENANCE aims to improve media and information literacy, partners are currently running workshops and evaluations with media and information literacy practitioners to (i) ascertain feedback for the refinement of the Verification Indicator and (ii) identify how it may be used by a range of practitioners (e.g. libraries, NGOs) to support the services they provide to citizens. The minimal user model has been defined for the Personalised Companion. In line with the above, the project has pursued a communication and dissemination plan that supports awareness among and engagement with key stakeholders as well as a revised exploitation plan.
To measure success, PROVENANCE has identified three indicators of progress on impact: endorsement by verification practitioners, endorsement by content creators, and improved ability to evaluate content credibility as a result of using PROVENANCE.
To understand if PROVENANCE can improve people’s ability to evaluate content, we conducted a baseline study with 290 participants to assess their ability to recognise disinformation news as well as legitimate news. In addition to providing a baseline against which we can assess the performance of PROVENANCE tools, this study allowed us to test suitable survey measures for assessing trust and credibility. The initial study found that participants who are relatively well informed and media literate can identify disinformation news, but struggle to accurately identify real news. This is an important baseline for current work with less informed and less media literate groups. It also underscores the importance of developing tools that go beyond identifying disinformation to also support recognition of high-quality content.
To advance and measure endorsement by verification practitioners and content creators, the project’s evaluation surveys have developed specific measures regarding perceptions of usefulness, intention to use, and open-questions on features that would increase usefulness and intention to use. Currently, this is being assessed with a wide range of media and information literacy practitioners (e.g. educators, librarians, civil society NGOs). Future evaluation studies will investigate the same measures with verification practitioners and content creators.
At the same time, PROVENANCE is monitoring best-practice recommendations by researchers, leading verification practitioners (e.g. First Draft) and adult literacy experts to ensure these principles are reflected in successive iterations of the plugin. In addition, PROVENANCE partners have undertaken significant communication and dissemination activities with relevant stakeholders. For example, a demo of the plugin was presented to the UN International Organisation for Migration, which is working to combat disinformation about migrants. A demo to the national conference of the Library Association of Ireland is upcoming.
To understand if PROVENANCE can improve people’s ability to evaluate content, we conducted a baseline study with 290 participants to assess their ability to recognise disinformation news as well as legitimate news. In addition to providing a baseline against which we can assess the performance of PROVENANCE tools, this study allowed us to test suitable survey measures for assessing trust and credibility. The initial study found that participants who are relatively well informed and media literate can identify disinformation news, but struggle to accurately identify real news. This is an important baseline for current work with less informed and less media literate groups. It also underscores the importance of developing tools that go beyond identifying disinformation to also support recognition of high-quality content.
To advance and measure endorsement by verification practitioners and content creators, the project’s evaluation surveys have developed specific measures regarding perceptions of usefulness, intention to use, and open-questions on features that would increase usefulness and intention to use. Currently, this is being assessed with a wide range of media and information literacy practitioners (e.g. educators, librarians, civil society NGOs). Future evaluation studies will investigate the same measures with verification practitioners and content creators.
At the same time, PROVENANCE is monitoring best-practice recommendations by researchers, leading verification practitioners (e.g. First Draft) and adult literacy experts to ensure these principles are reflected in successive iterations of the plugin. In addition, PROVENANCE partners have undertaken significant communication and dissemination activities with relevant stakeholders. For example, a demo of the plugin was presented to the UN International Organisation for Migration, which is working to combat disinformation about migrants. A demo to the national conference of the Library Association of Ireland is upcoming.