SCAPEAudio Quality AssuranceAn application of cross correlationJesper Sindahl NielsenThe State and University Library & MA...
SCAPE         The State and University Library• Large national collections   • Radio & television:      • More than 1.000....
SCAPE                 The Two Problems• Overlap   • Input: 2 Audio (wav) files   • Guarantee: They overlap within the last...
SCAPE                  Cross Correlation• Main component of our solutions• Well known technique (folklore by now)• An audi...
SCAPECross Correlation: Example           Samples              5/22
SCAPECross Correlation: Example           Samples              6/22
SCAPECross Correlation: Example           Samples              7/22
SCAPECross Correlation: Example           Samples              8/22
SCAPECross Correlation: Example           Samples              9/22
SCAPE          Cross Correlation: ExampleThis had the highestcorrelation, thusthe output is ’2’.                          ...
SCAPE                  Cross Correlation• Naively running this algorithm is slow   • Running time O(n2)• Using Fourier tra...
SCAPE                The Overlap problem• We have 15+ years of radio broadcast from DR  (Danish Radio) on tape in 2-hour c...
SCAPE                  The Overlap problem• Solutions?  • Find the longest suffix of the first file, that is a prefix of t...
SCAPE                The Overlap problem• We implemented the procedure  • It becomes quite simple when relying on FFT libr...
SCAPE              The Migration Problem• Over time file types become endangered   • When did you last listen to a ‘real a...
SCAPE         The Quality Assurance Problem• How can we be sure the content is the same?  • We need methods for doing QA f...
SCAPEThe Quality Assurance Algorithm                                    17/22
SCAPE        The Quality Assurance Algorithm• Split the two output files into blocks of ~5 seconds.• If all blocks have th...
SCAPE                QA algorithm Results• We needed a data set  • We do not actually have any migration errors (yet!)  • ...
SCAPE                QA algorithm Results• How did we do?   • The tool reported all the introduced errors   • It reported ...
SCAPE                       Summary• Overlap   • On average about 5 minutes pr file.   • Seems to be very accurate   • Can...
SCAPE                         Questions?                          Thank youCode can be found at:https://github.com/openpla...
Upcoming SlideShare
Loading in …5
×

Audio Quality Assurance. An application of cross correlation

1,346 views
1,216 views

Published on

Jesper Sindahl Nielsen, State and University Library, Denmark, presented algorithms for automated quality assurance on audio files in context of preservation actions and
access. Cross correlation is used to compare the soundwaves.
In: iPRES 2012 – Proceedings of the 9th International Conference on Preservation of Digital Objects. Toronto 2012, 144-149.
ISBN 978-0-9917997-0-1

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,346
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Audio Quality Assurance. An application of cross correlation

  1. 1. SCAPEAudio Quality AssuranceAn application of cross correlationJesper Sindahl NielsenThe State and University Library & MADALGOiPRESToronto, 2012 This work was partially supported by the SCAPE Project. The SCAPE project is co-funded by the European Union under FP7 ICT-2009.4.1 (Grant Agreement number 270137).
  2. 2. SCAPE The State and University Library• Large national collections • Radio & television: • More than 1.000.000 hours – app. 1Pbytes of data • Web archive • More than 8 billion pages – app. 300Tbytes of data • Up-coming newspaper digitization project • 32 million pages – app. 800Tbytes of data • Many other collections of almost any kind and size• Digital preservation challanges in large scale • Fits perfectly with the overall objectives of SCAPE 2/22
  3. 3. SCAPE The Two Problems• Overlap • Input: 2 Audio (wav) files • Guarantee: They overlap within the last 6 minutes • Output: The exact timestamp where the overlap starts• Quality Assurance of Migration • Input: 2 Audio (wav) files • Guarantee: Playback of the same file with different players • Output: Whether the two files are ‘enough’ alike 3/22
  4. 4. SCAPE Cross Correlation• Main component of our solutions• Well known technique (folklore by now)• An audio (wav) file is just a function (signal) • At sample 1 we have an amplitude, f(1) • At sample 2 we have an amplitude, f(2) • ... At sample n, we have an amplitude, f(n)• What it does: Given two functions it computes how much to shift one function along the x-axis such that they have highest correlation. 4/22
  5. 5. SCAPECross Correlation: Example Samples 5/22
  6. 6. SCAPECross Correlation: Example Samples 6/22
  7. 7. SCAPECross Correlation: Example Samples 7/22
  8. 8. SCAPECross Correlation: Example Samples 8/22
  9. 9. SCAPECross Correlation: Example Samples 9/22
  10. 10. SCAPE Cross Correlation: ExampleThis had the highestcorrelation, thusthe output is ’2’. Samples 10/22
  11. 11. SCAPE Cross Correlation• Naively running this algorithm is slow • Running time O(n2)• Using Fourier transforms it is much faster • Running time O(n log n)• Any text book on signal processing will describe this procedure. • a short summary can be found in the paper as well 11/22
  12. 12. SCAPE The Overlap problem• We have 15+ years of radio broadcast from DR (Danish Radio) on tape in 2-hour chunks.• Recorded using 2 tape recorders. • Overlap occurs• Then digitized• Situation: Someone wants to listen to a program that spans two tapes (files) • Don’t want to listen to the same clip twice 12/22
  13. 13. SCAPE The Overlap problem• Solutions? • Find the longest suffix of the first file, that is a prefix of the second file (excluding meta data etc) – bitwise comparison. • Does not work. Audio files sounding the same do not necessarily have the same bit pattern. • Fingerprinting techniques • Seems excessive. • Some of them even calculate correlation as a subroutine. • Cross Correlation • It finds exactly what we want. • Cut out the last 6 minutes of the first file and the first 6 minutes of the second file, use cross correlation on the two clips. 13/22
  14. 14. SCAPE The Overlap problem• We implemented the procedure • It becomes quite simple when relying on FFT libraries • So it relies on FFTW (“Fastest Fourier Transform in the West”)• Results • Has been run on approximately 3 months of broadcasts • Around 1000 files, took around 85 hours • Found errors in the collection (missing files, wrong channel .. ~3%) • The rest has been nicely cut (91% of the 3 months) • 5 minutes pr overlap, including cutting the files and using a Quality Assurance check 14/22
  15. 15. SCAPE The Migration Problem• Over time file types become endangered • When did you last listen to a ‘real audio’ clip? • How many ‘gif’ images do you encounter today versus 5 or 10 years ago?• We still want to be able to hear/view them in fifty years• Solution: Migrate to a different, more preservation friendly format. • Real Audio  WAV files 15/22
  16. 16. SCAPE The Quality Assurance Problem• How can we be sure the content is the same? • We need methods for doing QA for audio • Simple methods: Is the length the same before and after? • Better: extract more sophisticated features from the content. • Do they have the same average sound level? • Our suggestion: Use two different migration programs and cross correlate the output 16/22
  17. 17. SCAPEThe Quality Assurance Algorithm 17/22
  18. 18. SCAPE The Quality Assurance Algorithm• Split the two output files into blocks of ~5 seconds.• If all blocks have the same offset, within a fixed parameter, we conclude the two files are equal, and the migration went as it should• Otherwise report an error 18/22
  19. 19. SCAPE QA algorithm Results• We needed a data set • We do not actually have any migration errors (yet!) • Solution: make an artificial data set • Some files turned in to complete garbage (except header) • Some files had some parts replaced by garbage • Rest were OK • The data set was 70 file pairs where • 3 files where complete garbage • 5 files where partly garbage 19/22
  20. 20. SCAPE QA algorithm Results• How did we do? • The tool reported all the introduced errors • It reported false positives • These can be removed by tweaking parameters• How long did it take? • 70 files – 4 hours and 45 minutes (each file is 2 hours). • ~4-5 minutes pr case• How much memory does it use? • Almost nothing (less than 10mb, no matter the input size) 20/22
  21. 21. SCAPE Summary• Overlap • On average about 5 minutes pr file. • Seems to be very accurate • Can we do it faster? – integrate with SCAPE platform?• Migration • On average about 5 minutes pr case • Caught the errors we thought of • Which errors do actually occur?• Can we apply these ideas in other contexts? • Video? Images? Is it too slow for this? 21/22
  22. 22. SCAPE Questions? Thank youCode can be found at:https://github.com/openplanets/scape-xcorrsoundA blog post about xcorrSound can be found onhttp://www.openplanetsfoundation.org/blogs/ 22/22

×