Dyspoissometer source code in C and Mathematica is here for the impatient. Documentation follows.
This work evolved into a protracted research project funded by Tigerspike that almost got me fired. Without the pioneering mentality and superhuman patience of Luke Janssen, Oliver Palmer, Stuart Christmas, and Steven Zhang, you would not have this tool. They even granted me the copyright when I left on amicable terms so I could continue to manage it. If you want to build a custom app around it, give them a call.
Dyspoissonism (dis - PWAH - suhn - izm) is a quantity in the range of zero to one which measures the maximum possible compression fraction achievable under conditions of lossless combinatorial data compression applied to an unbiased random list of symbols. Put another way, dyspoissonism is a robust metric of distributed (global) randomness, with lesser values corresponding to greater randomness. Part of the mission of this blog is to mathematically justify this assertion. The other goal is to teach you how to apply it in practice, on real world digital signals, using Dyspoissometer, an open source discrete statistical analysis tool available in C and Mathematica languages.
Over the centuries, various tools have been developed to allow us to quantify the error in various measurements, such as the Gaussian, Maxwell-Boltzmann, and Poisson distributions familiar to most scientists and engineers. Loosely speaking, wide fuzzy distributions correspond to more information and narrow spikey distributions correspond to less information.
Bear in mind, what humans perceive as "interesting" is, from a statistical perspective, generally quite sparse in information content. This is the fundamental assertion behind the field of compressed sensing. Thus the narrower distributions are regarded as more "informative", which is reflective of their capacity to offer us insights into the operation of the universe -- not to provide us with a huge quantity of incomprehensible information.
It is entirely too common that analog statistical methods are (sometimes grossly) misapplied to discrete systems. Unfortunately, there is a dearth of tools appropriate for such ends. But they say that necessity is the mother of invention. So thanks again to Tigerspike, I'm pleased to offer you the following tutorial on dyspoissonism in addition to the source code linked above. You will find other articles posted to this blog from time to time, but here are the basics:
1. The Terminology of Mask Lists
2. Way Count: Counting the Number of Configurations
3. The Logfreedom Formula
4. Computing the Logarithm of a Factorial
5. Bits, Eubits, Logfreedom, and Entropy
6. Dyspoissonism Defined
7. Random Number Generators and Logfreedom
8. Information Sparsity vs. Dyspoissonism
9. Rapid Approximation of Maximum Logfreedom
10. Dyspoissometer Source Code
11. Dyspoissometer Demo Output
This work evolved into a protracted research project funded by Tigerspike that almost got me fired. Without the pioneering mentality and superhuman patience of Luke Janssen, Oliver Palmer, Stuart Christmas, and Steven Zhang, you would not have this tool. They even granted me the copyright when I left on amicable terms so I could continue to manage it. If you want to build a custom app around it, give them a call.
Dyspoissonism (dis - PWAH - suhn - izm) is a quantity in the range of zero to one which measures the maximum possible compression fraction achievable under conditions of lossless combinatorial data compression applied to an unbiased random list of symbols. Put another way, dyspoissonism is a robust metric of distributed (global) randomness, with lesser values corresponding to greater randomness. Part of the mission of this blog is to mathematically justify this assertion. The other goal is to teach you how to apply it in practice, on real world digital signals, using Dyspoissometer, an open source discrete statistical analysis tool available in C and Mathematica languages.
Over the centuries, various tools have been developed to allow us to quantify the error in various measurements, such as the Gaussian, Maxwell-Boltzmann, and Poisson distributions familiar to most scientists and engineers. Loosely speaking, wide fuzzy distributions correspond to more information and narrow spikey distributions correspond to less information.
Bear in mind, what humans perceive as "interesting" is, from a statistical perspective, generally quite sparse in information content. This is the fundamental assertion behind the field of compressed sensing. Thus the narrower distributions are regarded as more "informative", which is reflective of their capacity to offer us insights into the operation of the universe -- not to provide us with a huge quantity of incomprehensible information.
It is entirely too common that analog statistical methods are (sometimes grossly) misapplied to discrete systems. Unfortunately, there is a dearth of tools appropriate for such ends. But they say that necessity is the mother of invention. So thanks again to Tigerspike, I'm pleased to offer you the following tutorial on dyspoissonism in addition to the source code linked above. You will find other articles posted to this blog from time to time, but here are the basics:
1. The Terminology of Mask Lists
2. Way Count: Counting the Number of Configurations
3. The Logfreedom Formula
4. Computing the Logarithm of a Factorial
5. Bits, Eubits, Logfreedom, and Entropy
6. Dyspoissonism Defined
7. Random Number Generators and Logfreedom
8. Information Sparsity vs. Dyspoissonism
9. Rapid Approximation of Maximum Logfreedom
10. Dyspoissometer Source Code
11. Dyspoissometer Demo Output
No comments:
Post a Comment