endless compression

[markw at mohawksoft.com (markw at mohawksoft.com)]

>
> Hi Mark,

>
> Since its not relevant to this discussion, lets ignore the origin
> (whether its a PRNG [1] or some other source) of your n-bit streams and
> concentrate on the important aspect which is the encoding.  Lets imagine
> that you have enough of these bit streams (in your case, PRNGs) to
> represent every possible combination of n bits (which is 2^n streams).
> You then need a way of *remembering* which one of the 2^n streams should
> be used when you decompress.  And to uniquely represent each of the 2^n
> streams, you will need, at an absolute minimum, an n-bit index.  So, in
> the general case of trying to compress every possible input stream, what
> have you gained?
>
> Absolutely nothing.
>
> The above argument can be extended to variable-length streams and I
> leave that as an exercise for you to work out.

The theory is that it is NOT every possible combination of bits. It is a
stream of bits that we know to be less likely then the universe of its
range. It is known to be an already compressed stream. This means it does
not have many, if any, repeating or recognized patterns.

Because we know the stream patterns are not *ALL* the 2^n possible orders,
but (2^n)-(Z) compressability depends on the value of Z. Z is the number
of recognizable patterns removed from the original stream.

If the value of Z is sufficiently large, you can have compressability.

You are missing idea that it is *not* pure math. This is like the debate
between chaos theory and traditional physics, Einstein's quote "God does
not play dice." is a prime example. It is about probabilities. If given a
long enough random stream, some section of it will fit a segment.

Again, I can't believe that I have to keep saying this, I do not believe
that this works or that it doesn't, but I refuse to dismiss it saying "It
can't work" based on a dogma that that has so often been shown to be
wrong.
>
> Its interesting to note that essentially all of the great (fundamental)
> discoveries in math and science tend to be "limiting" in the sense that
> they place some constraint on what can or can't be achieved.  For
> instance:
>
>  - speed of light (the propagation of information)

Has been observed to be broken in experiments with photons and neutrinos.

>  - 2nd law of thermodynamics (the "arrow of time")
>  - Cramer-Rao Lower Bound (statistics)
>  - Planck constant (Heisenberg and the ability to resolve space-time)
>  - Information Theory (Shannon and maximum transmission rates)
>
> These are not the nonsense limits ("couldn't travel faster than 60 MPH")
> that you mention.  In particular, the last (Shannon) is one that you
> really ought to look into since it should be required reading for anyone
> playing with compression techniques.

This isn't a transmission issue, it is a study of math and probability.
Before computers, mathmaticians didn't believe that non terminating non
repeating equasions existed.

Are you saying that if you have an infinite amount of CPU and an infinite
amount of disk space on each in the form of data dictionaries and codec
routines, it would never be possible?