Is compression breakthrough too good to be true?
By Sam Costello
January 10, 2002 3:12 pm PT
ZEOSYNC ON MONDAY achieved something that mathematicians and computer scientists have claimed for 50 years is impossible: multi-pass, 100:1 compression of files without losing any data.
If it actually works, West Palm Beach, Fla.-based ZeoSync's system could compress files multiple times, making the files up to 100 times smaller than the original, and without losing any data, which is unavoidable with many current compression schemes.
But since the announcement, PhDs and analysts have said, often forcefully, that ZeoSync is wrong. ZeoSync, however, promises that proof -- in the form of demonstrations and products -- will begin to appear soon.
If ZeoSync is right and it has achieved this breakthrough, a revolution in storage and communication could be in the offing, according to analysts and those involved in the work. With this sort of compression, video-on-demand and other high-bandwidth applications could be available over standard phone line modems and wouldn't require broadband connections, according to David Hill, research director of storage and storage management at the research firm Aberdeen Group Inc. But, many long-time observers of compression technology say, seemingly too-good-to-be-true advances in the field have come -- and gone unsuccessfully -- before.
The details of how exactly ZeoSync's technology works are murky. In its press release announcing the "breakthrough", the West Palm Beach, Florida-based company provided a complex explanation, saying that the technology "intentionally randomizes naturally occurring patterns to form entropy-like random sequences" and then "encodes these singular-bit-variance strings within complex combinatorial series to result in massively reduced ... equivalents." The technology would be included in chips, for encoding and decoding, the company said.
When asked to explain what that meant in layman's terms, ZeoSync Chairman and Chief Executive Office Peter St. George provided an explanation that would, no doubt, confuse a layman: That the company's technology takes data files and "creates multidimensional constructs" out of them.
Jim Dyer, the Arkansas director of Radical Systems, who also worked on the project doing both testing and development, explained that unlike traditional compression schemes that seek to remove redundancy to achieve their compression, ZeoSync's technology attacks data packets as a whole, allowing them to be compressed more than once. Standard compression schemes can only compress files once. Radical has worked as a subcontractor with ZeoSync for about two and a half years on this project, Dyer said.
Though ZeoSync has briefed a number of analysts about its technology, the company hasn't shown them enough details to substantiate its claims, said Aberdeen's Hill. Hill, who was briefed last week, said that he "(doesn't) know enough (about the technology) to be able to determine feasibility" because ZeoSync did not provide a working demo.
Other analysts take a dimmer view of ZeoSync not presenting more data. Eric Scheirer, a senior analyst at Forrester Research who was briefed on the technology on Tuesday, said "all of their materials are hidden beneath a very thin layer of obfuscation" and that when he pressed them for more details, their terminology became fuzzy. He also said that the company claimed a number of details were covered under its "proprietary" methods and therefore could not discuss them.
Talking to ZeoSync was "a lot of going around in circles," he said.
"There's absolutely no chance that ZeoSync has accomplished what they've claimed," said Scheirer, who holds a PhD in audio compression from the Massachusetts Institute of Technology and has published more than a dozen peer-reviewed papers on the subject. "It's simply impossible."
ZeoSync's St. George, however, has heard the skeptics and says that they are wrong. St. George, who has been working on the research for 12 years -- just over two of those years with ZeoSync -- claims that what his company says it has done is possible "because we took an approach that took us out of the box that everyone was living in."
"People want to define themselves by limitations," he said. "(People) want to put (themselves) in a small tank because then we're a big fish."
St. George certainly does come from outside the box of the traditional mathematician. He holds no college degree, though he says he did attend both Syracuse University and the University of Utah for three years each. He is entirely self-taught when it comes to mathematics, he says.
Those that are skeptical, but not dismissive of ZeoSync's claims, such as Aberdeen's Hill, urge ZeoSync to offer more proof than it has to appease them. (ZeoSync declined to immediately offer any further proof of its claims to the IDG News Service as well). The company should allow other mathematicians to examine its data and conduct tests, Hill said. The company should also offer a more public demonstration of the technology, he said.
Having a demonstration "would be a very useful thing," he said. As ZeoSync has not yet made one, however, "I probably would have waited a little longer (to announce)," Hill added.
"When you claim a breakthrough that is this important, the world has to look at it with skepticism" and that skepticism is only eliminated when the work is validated by other respected people in the field, he said.
St. George turns back these suggestions as well.
"If we sat around and published nothing but scientific journals and white papers ... the people of the world won't receive the benefits of the technology," he said.
"For every person who says it might not work, there are 10 saying it does," he said, adding that he's received many congratulatory e-mails since the announcement.
St. George said that scientists and researchers are discounting his claims because "they have vested interest in the status quo ... their lives' works are associated with these things." St. George also changed course for the company and said that a public demonstration of the technology would be announced next week.
"I would love it if their claims were true, but their claims aren't true and there is no way their claims are true," Forrester's Scheirer said.
As part of the evidence backing the claims, St. George and ZeoSync touted their association with dozens of top mathematicians at colleges and universities throughout the world, including Harvard University, MIT, Stanford University and the University of California at Berkeley. Some of the mathematicians that they name, however, aren't quite as enthusiastic.
John Post, an assistant professor of Electrical Engineering at Arkansas Tech. University, is touted as one of the scientists working with ZeoSync on the project, but in an e-mail to the IDG News Service said that "I haven't been a witness to this technology, so personally (I) can't make any claims on the behalf of ZeoSync." Post did allow, though, that ZeoSync's claims may be possible on certain data. Post said he has been working with ZeoSync as a consultant, in what he described as a "small role," since Dec. 2001.
Steve Smale, an award-winning researcher and professor emeritus at the University of California at Berkeley, was also named as someone involved in the project by ZeoSync. Smale said in an e-mail to the IDG News Service that he had only spent "one hour" on the project and was "in no position to say anything about these claims."
Another of the researchers that ZeoSync said was involved, Polish mathematician Wlodzimierz Holsztynski, posted a message to the Polish pl.sci.matematyka newsgroup after ZeoSync's announcement and use of his name saying that he had asked ZeoSync not to use his name in any of their materials and that "I have more important successes than those published on (ZeoSync's) page."
When asked to comment on Holsztynski's message, St. George said that ZeoSync has complied with Holsztynski's request to not have his biography published on ZeoSync's Web site , but that he is employed as a consultant. He also discounted the idea that any of the three scientists were in a position to the substantiate ZeoSync's claims.
"These people are not substantiating these claims," but rather have all worked on discreet areas of the technology, St. George said. "There is only a very small number of scientists who have worked on all components (of the technology)," he said.
"This is a very big announcement and it takes very broad shoulders" to stand by it, something he is willing to do, he said.
Forrester's Scheirer allows that ZeoSync could simply be mistaken about its results or may have created a technique that works under very limited conditions. "Breakthroughs" in compression technology have been announced before, though, and they "always turn out to not pan out," he said.
But those who are convinced, remain convinced.
Radical Systems' Dyer, who has worked on the project and claims to have seen it work, isn't dissuaded by skeptics. Dyer agrees with St. George that taking a non-standard approach, and believing that something can be done even when others say it can't, seems to have paid off in this instance.
Dyer also cites the advances in computer processing power in the last few years as possibly playing a role in the breakthrough.
"Without the current computer horsepower, we didn't have a way to test things like this," he said, noting that the same thing is true of other advanced computing applications such as quantum computing.
ZeoSync's St. George remains optimistic as well. He claims that the company has been in discussions about its technology with Hollywood studios and chip companies and that those discussions will lead to announcements in about six months. He also promises that Web demonstrations of the technology are forthcoming.
"We understand that (a lack of public proof is) the problem, we're not hiding from it," he said. "Once the technology is demonstrated, it will stand on its own, and (then) let people say what they will."
"I will not flinch at ... marching this onto the global stage," he said. "It's just a matter of time before (the skeptics) see."
If all goes as planned, within a few weeks' time and whatever the outcome, the skeptics will indeed see.
Sam Costello is a Boston-based correspondent for the IDG News Service, an InfoWorld affiliate. Olo Sawa, of Computerworld Polska, contributed to this report.
100:1 compression.. is it possible ?
Reply With Quote
But like I said, very limited on acual usefulness, because a file that was all 0101010 would actually end up being larger when you ziped it.