Analysis of Lempel-Ziv'78 for Markov Sources - Archive ouverte HAL Access content directly
Conference Papers Year :

Analysis of Lempel-Ziv'78 for Markov Sources

Abstract

Lempel-Ziv'78 is one of the most popular data compression algorithms. Over the last few decades fascinating properties of LZ78 were uncovered. Among others, in 1995 we settled the Ziv conjecture by proving that for a memoryless source the number of LZ78 phrases satisfies the Central Limit Theorem (CLT). Since then the quest commenced to extend it to Markov sources. However, despite several attempts this problem is still open. The 1995 proof of the Ziv conjecture was based on two models: In the DST-model, the associated digital search tree (DST) is built over m independent strings. In the LZ-model a single string of length n is partitioned into variable length phrases such that the next phrase is not seen in the past as a phrase. The Ziv conjecture for memoryless source was settled by proving that both DST-model and the LZ-model are asymptotically equivalent. The main result of this paper shows that this is not the case for the LZ78 algorithm over Markov sources. In addition, we develop here a large deviation for the number of phrases in the LZ78 and give a precise asymptotic expression for the redundancy which is the excess of LZ78 code over the entropy of the source. We establish these findings using a combination of combinatorial and analytic tools. In particular, to handle the strong dependency between Markov phrases, we introduce and precisely analyze the so called tail symbol which is the first symbol of the next phrase in the LZ78 parsing.
Fichier principal
Vignette du fichier
LIPIcs-AofA-2020-15.pdf (704.83 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03139593 , version 1 (12-02-2021)

Identifiers

Cite

Philippe Jacquet, Wojciech Szpankowski. Analysis of Lempel-Ziv'78 for Markov Sources. AofA2020 - 31st International Conference on Probabilistic, Combinatorial and Asymptotic Methods for the Analysis of Algorithms, Jun 2020, Klagenfurt, Austria. ⟨10.4230/LIPIcs.AofA.2020.15⟩. ⟨hal-03139593⟩
29 View
30 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More