I am surprised looking at the speed of most AV scanners. Let us say there is a 1 MB file which may potentially have one or more of 89000 viruses. All their signatures have to be searched in the entire content of the 1 MB file, as I understand it. Then, how is this done so fast, where the 1 MB file is scanned in less than a second?
Hope I could get hold of an opensource AV solution so that I could read the code to understand the excellence behind.
Any information on this topic will be helpful.
April 24th, 2006, 07:27 AM
Hi kautilya, if you think virus scanners are fast then just take a test of the open-source AV called "clamwin". It's really slow, I think it really does scan every file in its entirety.
Why I think that most AV's are fast is that they are probably smart enough to exclude many files from requiring a complete scan. I would guess that with certain file types (say a wav file for example) it would be sufficient to just verify that the file-header matches the file type correctly and then just declare it safe without further scanning.
April 24th, 2006, 12:20 PM
It works something like this:
1. There are probably about 350,000 malwares by now.
2. A lot of them are obsolete and will not run on a modern system.
3. A lot of them will only work in a certain way, so an executable file infector is useless with an MP3 file................and so on.
What is happening is that your file types are getting scanned for known infection types that suit that file. That cuts the scan down a lot.
Also, look at if you are doing "deep scanning", "heuristic scanning", and if you arescanning within "compressed/archived" files?
Those options significantly change your response time. Anyways, how cam Google give me 60 Million results in under two seconds?
April 27th, 2006, 02:29 PM
With google I think it's different. They already have an index of keywords and stuff like that, and this cuts the time tremendously. It's not like they crawl sites only when you serach for something. They do it before and prepare an index of keywords and based on this index your results are displayed. One can try searching for some akward phrase or word and it will turn up zero matches. The same is true fro certain misspelt words.
Plz correct me If I'm wrong.
April 27th, 2006, 02:53 PM
Hi darkcod3r ,
Yes, that is true, but somewhat similar to the AVs "pattern" or "dat" files which are preconstructed.
You can also use "quick" or "intelligent" scanning options as well. Here you know if a malware is "stand alone" or an "infector". Obviously if it is stand alone you would only have to scan a very little to recognise it. In the case of the infector (virus) you would know if it replaced a valid file, appended or prepended itself or inserted itself.
Using that knowledge you can minimise the amount of the file that needs to be scanned. Also you would scan only "favourite" targets like executables, and tend to ignore e-mails, pictures, music and the like which would only get looked at in a full scan. You might particularly take this option if you are interactively scanning downloaded items anyway.
Another method is to calculate checksums for the files. The scan would then only look at files that had changed since the last checksums were set, or that were new.
Obviously setting your AV to do a full scan with updated pattern files, heuristics turned on and running in safe mode would be the most reliable.