Periodic Reporting for period 3 - BISON (BIg Speech data analytics for cONtact centres)
Reporting period: 2017-01-01 to 2017-12-31
According to the latest European Contact Center Benchmark data, the European Contact Center (CC) industry involves more than 35,500 contact centers with 3.8 Million jobs in 30 countries (a median reported size is 81 positions). The sectors keeps growing structurally at an annual pace of 3,6% in employment. A usual CC operation generates a wealth of spoken data. A typical contact center with 1,000 agents, each doing 40 calls a day with an average call lasting for 3 minutes, generates 2,000 hours of audio every 24 hours. This data is the core of CC’s business; however, its current exploitation is rather limited.
The objective of BISON is to create a multi-lingual, modular and highly versatile software system for big speech data analytics in contact centers targeting:
1. basic speech data mining technologies
2. transforming the basic data into information valuable for business strategies
3. real-deployment of the systems by real CCs
The objective of BISON is to create a multi-lingual, modular and highly versatile software system for big speech data analytics in contact centers targeting:
1. basic speech data mining technologies
2. transforming the basic data into information valuable for business strategies
3. real-deployment of the systems by real CCs
The project was structured into eight workpackages and the progress and results are reported per WP.
WP1 dealt with project management in a consortium of 8 partners from 5 European countries. During the project execution, no significant obstacles were encountered and the consortium did not change. WP1 was responsible for contractual handling among partners and for the usual project activities including physical annual reviews (BISON opted for 12-month review cycle), physical meetings (3-4 per year) and teleconferences (over 20 each year of the project). Together with WP8 (legal and ethical issues), WP1 ensured contractual handling: in addition to the usual consortium and grant agreement.
WP2 dealt with user requirements and user testing of the developed technologies. At the very beginning of the project, deliverable D2.1 gathered initial user requirements that were further precised during the project. The user testing started early after the release of the first project demonstrator (smallBison, in WP6). In the evaluations of D2.2 the feedback was gathered from the Contact center (CC) partners and included in the development cycle.
WP3 dealt with the main asset necessary to any analytics development - the data. BISON relied partly on data available from standard commercial resources, but the main focus was on collection of real CC data with anonymization and manual annotations. The final status of CC data collection is 530 hours of audio accounting for 261 hours of annotated speech. In addition, a total of 1880 hours of unannotated audio was collected. WP3 also dealt with the collection of public data (including solving legal issues of their use, in cooperation with WP8), and annotation data for business data mining. A small set of simulated CC data was collected and is available for the research and commercial use from project’s web-site.
WP4 dealt with all aspects of speech data mining. Its research part brought significant advances in automatic transcription, speaker recognition and language recognition. The common denominators of this work was deep learning, massive use of DNN embeddings, multi-lingual and unsupervised training and data augmentation. The production part of WP4 concentrated on generating models for BISON production ASR systems. We have come up with models for 14 European languages. A special case was Luxembourgish (an under-resourced language) where a system with decent performance could be obtained thanks to unsupervised and cross-domain training.
WP5 concentrated on business data mining on the top of speech analytics. D5.1 contained initial plans for extracting useful business information (such as Key Performance Indicators) from speech. These plans were further precised and turned into reality in tight cooperation of WP5 and WP4. Significant efforts were devoted to the estimation of Customer Satisfaction from speech based on Convolutional Neural Networks and we have also worked on estimation of agent performance, call taxonomy, and work-flow monitoring (D5.3). WP5 (in combination with WP6) also included visualization of business results in reconfigurable dashboards. WP5 also included work on indexing and fast database access to data (D5.2).
WP6 dealt with integration and production of two prototypes: smallBison (after the 1st year of project) and bigBison (final prototype). Already in D6.1 APIs and specifications were drafted that gave a clear framework to the development teams. smallBison demonstrated the basic funcitonality of speech data mining and its different versions were evaluated by CC users (WP2). The last year of project witnessed significant amount of work towards bigBison, that finally included all planned functionalities, from real-time speech data processing, through speaker recognition, to integrability with 3rd party CC software solutions.Besides small and bigBison, WP6 also dealt with the software framework for CC data collection and annotation (D6.3) and reconfigurable software environment for visualizaiton of results (together with WP5).
WP7 dealt with exploitation and dissemination issues of the project. As this was an innovation action aiming at bringing the results to the market, significant effort was devoted to business planning (D7.[234]). The principles of commercialization were defined and in cooperation with WP8, legal framework for the exploitation of results was laid. WP7 also included dissemination to professional public, participation at EC events and general public dissemination in all traditional and electronic media. The consortium was active in academic publishing: 28 publications include top scientific conferences, two journal papers, and a prize at the international Interspeech conference. WP7 also maintained project web-page (D7.1) and social sites.
WP8 dealt with legal and ethical issues underpinning the technology development, from data, through models, to usage scenarios. D8.[243] was a series of documents supporting the team in all the project work. WP8 work culminated in the final WP8 deliverables: D8.5 is intended both for the BISON partners and for the wider public interested in speech data analytics and data protection. It offers an inter-disciplinary design & development methodology for creating a law-abiding system, where the legal requirements – in particular the GDPR - are embedded from the earliest design stage. D8.6 provides with specific guidelines and recommendations to BISON developers and end users, in a complete view aimed at covering all stages.
WP1 dealt with project management in a consortium of 8 partners from 5 European countries. During the project execution, no significant obstacles were encountered and the consortium did not change. WP1 was responsible for contractual handling among partners and for the usual project activities including physical annual reviews (BISON opted for 12-month review cycle), physical meetings (3-4 per year) and teleconferences (over 20 each year of the project). Together with WP8 (legal and ethical issues), WP1 ensured contractual handling: in addition to the usual consortium and grant agreement.
WP2 dealt with user requirements and user testing of the developed technologies. At the very beginning of the project, deliverable D2.1 gathered initial user requirements that were further precised during the project. The user testing started early after the release of the first project demonstrator (smallBison, in WP6). In the evaluations of D2.2 the feedback was gathered from the Contact center (CC) partners and included in the development cycle.
WP3 dealt with the main asset necessary to any analytics development - the data. BISON relied partly on data available from standard commercial resources, but the main focus was on collection of real CC data with anonymization and manual annotations. The final status of CC data collection is 530 hours of audio accounting for 261 hours of annotated speech. In addition, a total of 1880 hours of unannotated audio was collected. WP3 also dealt with the collection of public data (including solving legal issues of their use, in cooperation with WP8), and annotation data for business data mining. A small set of simulated CC data was collected and is available for the research and commercial use from project’s web-site.
WP4 dealt with all aspects of speech data mining. Its research part brought significant advances in automatic transcription, speaker recognition and language recognition. The common denominators of this work was deep learning, massive use of DNN embeddings, multi-lingual and unsupervised training and data augmentation. The production part of WP4 concentrated on generating models for BISON production ASR systems. We have come up with models for 14 European languages. A special case was Luxembourgish (an under-resourced language) where a system with decent performance could be obtained thanks to unsupervised and cross-domain training.
WP5 concentrated on business data mining on the top of speech analytics. D5.1 contained initial plans for extracting useful business information (such as Key Performance Indicators) from speech. These plans were further precised and turned into reality in tight cooperation of WP5 and WP4. Significant efforts were devoted to the estimation of Customer Satisfaction from speech based on Convolutional Neural Networks and we have also worked on estimation of agent performance, call taxonomy, and work-flow monitoring (D5.3). WP5 (in combination with WP6) also included visualization of business results in reconfigurable dashboards. WP5 also included work on indexing and fast database access to data (D5.2).
WP6 dealt with integration and production of two prototypes: smallBison (after the 1st year of project) and bigBison (final prototype). Already in D6.1 APIs and specifications were drafted that gave a clear framework to the development teams. smallBison demonstrated the basic funcitonality of speech data mining and its different versions were evaluated by CC users (WP2). The last year of project witnessed significant amount of work towards bigBison, that finally included all planned functionalities, from real-time speech data processing, through speaker recognition, to integrability with 3rd party CC software solutions.Besides small and bigBison, WP6 also dealt with the software framework for CC data collection and annotation (D6.3) and reconfigurable software environment for visualizaiton of results (together with WP5).
WP7 dealt with exploitation and dissemination issues of the project. As this was an innovation action aiming at bringing the results to the market, significant effort was devoted to business planning (D7.[234]). The principles of commercialization were defined and in cooperation with WP8, legal framework for the exploitation of results was laid. WP7 also included dissemination to professional public, participation at EC events and general public dissemination in all traditional and electronic media. The consortium was active in academic publishing: 28 publications include top scientific conferences, two journal papers, and a prize at the international Interspeech conference. WP7 also maintained project web-page (D7.1) and social sites.
WP8 dealt with legal and ethical issues underpinning the technology development, from data, through models, to usage scenarios. D8.[243] was a series of documents supporting the team in all the project work. WP8 work culminated in the final WP8 deliverables: D8.5 is intended both for the BISON partners and for the wider public interested in speech data analytics and data protection. It offers an inter-disciplinary design & development methodology for creating a law-abiding system, where the legal requirements – in particular the GDPR - are embedded from the earliest design stage. D8.6 provides with specific guidelines and recommendations to BISON developers and end users, in a complete view aimed at covering all stages.
In the BISON project, the most important progress beyond state-of the art was:
Creation of robust project demonstrator - bigBison - by easier backend structure
Evolution of language portfolio offering 13 European languages
New framework for data collection - anonymized features, business outcome data, public data and acted data sets are ready for demonstration.
Integration of real-time based technologies as speaker identification or keyword spotting
Business outcome mining, including customer satisfaction analysis and call taxonomy.
Creation of a re-configurable Dashboard tool for a comprehensive overview of speech data mining results through data visualization.
Specific legal and ethical guidelines and recommendations to BISON developers and end users, in a complete view aimed at covering all stages, from design and development to deployment and maintenance. The 360-degree overview is completed by a specific section with information targeted to data subjects
Creation of robust project demonstrator - bigBison - by easier backend structure
Evolution of language portfolio offering 13 European languages
New framework for data collection - anonymized features, business outcome data, public data and acted data sets are ready for demonstration.
Integration of real-time based technologies as speaker identification or keyword spotting
Business outcome mining, including customer satisfaction analysis and call taxonomy.
Creation of a re-configurable Dashboard tool for a comprehensive overview of speech data mining results through data visualization.
Specific legal and ethical guidelines and recommendations to BISON developers and end users, in a complete view aimed at covering all stages, from design and development to deployment and maintenance. The 360-degree overview is completed by a specific section with information targeted to data subjects