Your SlideShare is downloading. ×
20 MINNO (04): Lossless packet compression: a technique to deal with bandwidth limitations
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

20 MINNO (04): Lossless packet compression: a technique to deal with bandwidth limitations

968
views

Published on

Lossless packet compression: a technique to deal with bandwidth limitations

Lossless packet compression: a technique to deal with bandwidth limitations

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
968
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Lossless packet compression: atechnique to deal with bandwidthlimitationsDavid Evans
  • 2. The problem with housekeeping..  A mix of different field lengths and data types  Different packets are then written into the same packet storeThis mixed up data is hard to compress.
  • 3. Initial steps.. 1) Recreate a real on-board packet store on the ground ROSETTA 2) Compress it with commercial products to determine level of information redundancy RICE 100% winzip 27% 7zip 14% 3) Group packet types before compressing winzip 16% 7zip 9%
  • 4. A bit of lateral thinking.. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # . ## ## . . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# # # # .. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # # .. .. . . .#. # . . ## .# #. . . # . . .# .# # . # #. . #. ## .# . . # .. . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# . . # .. traditional fee . . .. #. .# . # . # ## .# .. .. . # .. #. .. .# # . . # # .. .# . d . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .. #. . . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .. ## . . .#. # . . ## .# #. . . # . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .. .. . . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .. .# . . .#. # . . ## .# #. . . # . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # # .. #. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # # .. ## . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## .# .. .. . # .. #. .. .# # . . # # .# .. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## .# .. .. . # .. #. .. .# # . . # # .# .# . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .# #. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .# ## . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .# .. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .# .# . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . # # .. transposed feed 7zip 9% .. ..#..#. # . ###....... # .# . # #..#.##.## # #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # .####. .. ..#..#. # . ###....... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . # . ..#.#.#....# #. . # #..... .. ..#..#. # . ###.#..... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # #...#. tra itio afed d nl e .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # #..#.. .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . # . ..#.#.#....# #. . # #..##. .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # #..... .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. . # . ..#.#.#....# #. . # #...#. .. ..#..#. # . ###....... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #..#.. winzip .. ..#..#. # . ###....... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #..##. .. ..#..#. # . ###.#..... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #.#... .. ..#..#. # . ###.#..... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #.#.#. .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #.##.. .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #.###. 10% .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #.#... .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #.#.#. tra p s df e ns oe e d
  • 5. Let’s try something really simple… 7zip 9% .. ..#..#. # . ###....... # .# . # #..#.##.## # #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # .####. .. ..#..#. # . ###....... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . # . ..#.#.#....# #. . # #..... .. ..#..#. # . ###.#..... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # #...#. tra itio afed d nl e .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # #..#.. .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. . # . ..#.#.#....# #. . # #..##. .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. . . . ..#.#.#....# #. . # #..... .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. . # . ..#.#.#....# #. . # #...#. .. ..#..#. # . ###....... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #..#.. winzip .. ..#..#. # . ###....... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #..##. .. ..#..#. # . ###.#..... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #.#... .. ..#..#. # . ###.#..... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #.#.#. .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #.##.. .. ..#..#. # . ####...... # .# . # #..#.##.#. . #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #.###. 10% .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # . . ..#.#.#....# #. . # #.#... .. ..#..#. # . #####..... # .# . # #..#.##.#. # #.. ..#. # . .##.##.. # # . ..#.#.#....# #. . # #.#.#. tra p s df e ns oe e d RLE 14%Our simple preprocessing and fax machine algorithm (less than a pageof C++ in total) was compressing ROSETTA housekeeping almost aswell as the best algorithm on the market
  • 6. Does this work on all missions? MISSION NAME MISSION TYPE COMPRESSION (% of original) Columbus Human Spaceflight 5.75 Rosetta Interplanetary 14.06 Venus Express Interplanetary 18.23 Proba-1 Technology Demo 25.93 Herschel Astronomy 28.00 Goce Earth Observation 38.20
  • 7. Parameter and bit level compressibility 160 140 120 100 80 60 40 20 0 1 107 213 319 425 531 637 743 849 955 1061 1167 1273 1379 1485 1591 1697 1803 1909 2015 2121 2227 Figure 3. Compressibility (% of original size) of each bit column for a packet that compresses well [CDMU Herschel Periodic P1 HK Parameter Report]
  • 8. Parameter and bit level compressibility 120 100 80 60 40 20 0 1 161 321 481 641 801 961 1121 1281 1441 1601 1761 1921 2081 2241 2401 2561 2721 2881 3041 3201 3361 3521 3681 3841 4001 4161 Figure 5. Compressibility (% of original size) of each bit column for a packet that compresses badly [Herschel SCM Mode TM]Major influencing factor is AOCS design, mission environment is less important
  • 9. Resistance from an unexpected quarter.. Impractical Risky Not worth it
  • 10. How much data do we need to storebefore compressing it? 100 90 80 70 % Original size 60 50 40 30 20 10 0 10 50 100 250 500 1000 5000 # Packets 48
  • 11. Will it run on real hardware? DANGER TEST IN PROGRESSThe bit transposition and RLE algorithms were recoded to respect space codingstandards and loaded to a real LEON2 processor in ESA-ESTEC.The algorithm compressed a 42.4 kilobytes file (from VEX) to 6.55 kilobytes (i.e.15% of the original) in 0.5 seconds.Implemented in one day, worked first time.
  • 12. Now let’s address the risk argument.. Impractical Risky Not worth it
  • 13. Too Risky The error correction on the space link works at frame level, i.e. either all the data in a frame is received or the full frame is lost. Compression in itself does not increase the risk of information loss ONLY IFThe information in each frame can be interpreted independently of the others.
  • 14. The simple beauty of RLE “If two identical bytes are followed by a third, insert a one byte counter of how many more repeats follow” “FF FF FF FF FF FF FF FF 1C” becomes “FF FF 06 1C”  Incredibly simple to compress and expand  Self contained – so no risk increase!
  • 15. The effect of bit transposition . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # . ## ## . . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# # # # .. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # # .. .. . . .#. # . . ## .# #. . . # . . .# .# # . # #. . #. ## .# . . # .. . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# . . # .. traditional fee . . .. #. .# . # . # ## .# .. .. . # .. #. .. .# # . . # # .. .# . d . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .. #. . . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .. ## . . .#. # . . ## .# #. . . # . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .. .. . . .#. # . . ## .# #. . . . . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .. .# . . .#. # . . ## .# #. . . # . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # # .. #. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## .. .. .. . # .. #. .. .# # . . # # .. ## . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## .# .. .. . # .. #. .. .# # . . # # .# .. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## .# .. .. . # .. #. .. .# # . . # # .# .# . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .# #. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## #. .. .. . # .. #. .. .# # . . # # .# ## . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . . # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .# .. . . .#. # . . ## .# #. . # . . . .# .# # . # #. . #. ## .# . # # .. . . .. #. .# . # . # ## ## .. .. . # .. #. .. .# # . . # # .# .# . . .#. # . . ## .# #. . # # . . .# .# # . # #. . #. ## .# . # # .. transposed feedInstead of losing rows, you now lose columns…..a small set of parameters overa longer time as opposed to all parameters over a short time.Is this a problem? If so, we can already state our maximum exposure….48 packets.
  • 16. But what about if I can’t afford to lose anything? Simply use the time and bandwidth released to send the frames twice or implement a closed loop.. Original RepeatThis compression technique does not increase mission risk and the resourcesit releases can be used to reduce risk far below that normally accepted.Those missions that are concerned about the risk of information loss mustcompress their housekeeping telemetry. Especially in contingency cases.
  • 17. One more… Impractical Risky Not worth it
  • 18. Time is money Operations costs are dominated by manpower. Manpower costs are dominated by time.The time saved by HKTM compression will reduce how long an operationtakes and open up choices for when it can be performed.This will cut reaction times, an important operational metric.The time saved will enable operational feedback loops not possible before.
  • 19. Ease the pain of packet design Preparation tasks can account for a significant proportion of the total operations cost. Part of that work is to carefully select parameters and their respective sampling rate in the packets. Do we really need What sample rate is this parameter in this needed? mode? A common solution is to design asynchronous events to flag significant changes. This sounds easy!
  • 20. How does compressing telemetry in this way help?It effectively removes all “compressible” parameters (vast majority) fromthe packet design trades. One can simply select as many of these parameters to be in the housekeeping as one wantsNo need to guess if these parameter histories might be needed one dayNo need to design and test asynchronous events for these parametersNo need to predict spacecraft behavior - to decide on trigger metrics
  • 21. Improving observabilityCan sample “compressible” parameters at high rates with a low impact onbandwidth usage. Richer, finer information on the ground to analyzeLess need to guess or extrapolate when trying to reconstruct eventsLess chance of missing important information due to low sampling rates(short abnormal behavior like spikes, transients, high frequency switching)More chance of discovering correlations between parameter behavior andimportant events or trends
  • 22. Is it compatible with the presentinfrastructure? Packet Store Packet extraction Bit tr ansposition RLE encoding Store in special “frame length” packets Compressed Packet Store Transfer to mis sion control centre using normal PUS services, CCSDS frames, coding, modulation etc Compressed packets routed to Pre-process or Router pre-processor for decompression Recovered packets pass ed to Mission Control miss ion control system for normal Sys tem processing
  • 23. The futureIt is fair to say that it is becoming a standard solution for future studies sectionat ESA/ESOC for Phase 0/A work. Baseline for PROBA-3 Identified as an enabling technology for the Mars Atmospheric Sample Return Mission ESA patent filed in the USA
  • 24. Conclusion The Bit-Transposition and RLE algorithm • Compresses housekeeping data very effectively • Is extremely simple • demonstrated good performance on target hardware • Requires little change to the present CCSDS/PUS based infrastructure • Requires just 48 stored packets as input • Does not increase the risk of information loss It will • Reduce ground station usage and increase mission return • reduce ground reaction times • enable more efficient use of operations manpower • reduce engineering costs in the preparation phase • increase spacecraft observability and telemetry flexibility • decrease mission risk
  • 25. Conclusion Thank you
  • 26. Backup: Comparison with RICE