Shrink Your File Format

Want to shrink the file format in your program?  Here’s a method for shrinking your numbers down.  If you write the number “127″ to a file, you are writing three bytes to the file (the ASCII character for each number).  This might not seem like a big deal, but in the long haul you can save a ton of space using the following method.  Write the character for 127 (\x7f, but printed as ).  Now read the file’s charachter and convert it into a number.  You just saved two bytes.  For writing numbers bigger than 256 (ASCII limit), write partials of numbers.  Break these numbers up in parts of two (589406830=58|94|06|83|0).  It might not be the most efficient way to shrink these numbers down, but it is easy to code and could save space in the long run.  If you want to shrink your numbers even more, convert them to hexadecimal before writing them to a file (hex was originally designed to write huge numbers).  Read these suckers back, feed it all into one string, and convert the string to an integer.  Congratulations on shrinking your file format.

Formats

Cracking formats doesn’t seem be too hard.  I figured out most of the simple 16-color bitmap this afternoon using a hex editor.  It’s pretty simple.  Create a bunch of files of a format of your choosing.  Open these up in a hex editor (my pick is XVI32, a great freeware editor), and take a look at how the file is structured.  Find out what numbers represent and changes that take place in the hex when you change the file (coloring pixels, changing width, etc.).  Then you can see what parts of the file do.  Find out what changing numbers represent, when you mess around with the files these will change.  You have to see how these changes will work in the file.  A description of file size might only take up one byte in one instance, but if the file gets bigger it might take up a few bytes.  Of course, most formats have been explored already, and in great depth.  I ran into a few problems with my hacking of the bitmap, because I was forgetting the whole issue of palettes and compression, but you can learn a lot about a file by looking at it in a hex editor.  Change things with the hex, and look at changes in the file.  When you discover functions of the hex, make sure you write it down.  Here’s some documentation I did on bitmaps.  I have a more full explanation of what I discovered on another computer, but here’s a template for your file:

–Bitmap File Info–
Byte Structure:
Header- 42|4D|C6
File Size- 3rd address and more, ab|cd=cd|ab
Height- 12th address
Width- 16th address
Pixels, bytes before final 3 bytes-Stores two pixels per byte. End of file starts at top-right.

Colors:
0- Black
1- Dark Red
2- Dark Green
–EOF– 

If you’re interested in the bitmap format there are some very good guides, here’s one that is very informative and useful if you need to use the bitmap format.  For more information on formats visit Wotsit, or if you want to get into reverse engineering see this excellent wikibook.

Cheers,

Ivan

Writing a Virus Scanner (Part 1 of 2)

Introduction:

Okay, I’m sure everybody here wants to get down and dirty with the world of viruses. I’m sure you all have bad memories with viruses and now is your chance to get some revenge. All right, so our scanner won’t be that strong. In fact it will only detect one virus. To make matters worse the “virus” is just a test virus used to test out antivirus software. Still, armed with this information you can learn to apply these examples to finding real viruses. In the first part of this mini-series I will show you the basic theory behind antivirus software, so that we can write our own little scanning script. Obviously we won’t have advanced features such as quarantining, but if you add to the program, it could actually do some fine work. You should be able to apply this theory to whatever language you program in. I’ll be using Python.

Theory:

The basic theory behind antivirus software is to detect viruses based
on their signatures, a hexadecimal string based on the contents of a file. First I will show you how we get this string from a known virus. Then in the next part of the series, I’ll show you how to put this information to good use. Basically you find the virus signature by dumping the file in hexadecimal. Hexadecimal is a base-16 system (decimal is a base-10 system) and teaching it to you is outside the reach of this tutorial. If you want to learn hexadecimal (not completely necessary, but definitely helps out) I recommend going here.

Note: We will not be using the 0×1F format, nor will we be using the $1F. We will be just writing 1F. These numbers are still hexadecimal.

Okay, for the next step your antivirus/antispyware utility might interfere with our activities if it has automatic protection features that scan downloads. We are going to download the industry standard eicar test file.

WARNING: I cannot be held responsible for misuse of antivirus software when using this file. If you aren’t comfortable using antivirus software, this tutorial is not for you.

Download a test file (not one of the ZIP ones). Now we need to dump the file into hexadecimal. There are many programs that can dump into hexadecimal, but I assume you want to do this quickly. If that is the case, please visit Online Hex Dump. Upload your file and there should be hex output. Remove the line numbers and the parts that aren’t hex (the parts at the end of the line). Your final output should be this:

58 35 4F 21 50 25 40 41 50 5B 34 5C 50 5A 58 35
34 28 50 5E 29 37 43 43 29 37 7D 24 45 49 43 41
52 2D 53 54 41 4E 44 41 52 44 2D 41 4E 54 49 56
49 52 55 53 2D 54 45 53 54 2D 46 49 4C 45 21 24
48 2B 48 2A

Now you need to unspace this code. You can do this manually, but I just wrote a nice little Python script to do it for me:

string=”58 35 4F 21 50 25 40 41 50 5B 34 5C 50 5A 58 35 34 28 50 5E 29 37 43 43 29 37 7D 24 45 49 43 41 52 2D 53 54 41 4E 44 41 52 44 2D 41 4E 54 49 56 49 52 55 53 2D 54 45 53 54 2D 46 49 4C 45 21 24 48 2B 48 2A”
string=string.replace(” “,”")
print string 

Here’s what the final signature looks like:

58354F2150254041505B345C505A58353428505E2937434329377D244549
4341522D5354414E444152442D414E544956495255532D544553542D46494
C452124482B482A

This makes a lot faster if you are serious about all this virus signature stuff. Now you have the virus signature (note it should all be on one line, the example about isn’t on one line, due to bad WordPress blog editing in Opera)! Don’t worry, you won’t have to do this for every virus, or most viruses. There are sites that provide free virus signatures. One site I found is run by Lightspeed Systems. Their signature provided for this test signature isn’t the full hex dump. It’s actually only about half of it. Checking that half would still find the virus, it would just return more false-positives. The theory behind our virus scanner is that it will hex dump the file and compare it to a known list of virus signatures. To detect more advanced viruses you will have to learn about polymorphic viruses, which is definitley outside the scope of this tutorial.  Good luck. Learn all this so you’ll be ready for the next tutorial. If you don’t wish to write your own virus scanner, maybe you would like to help out with an open source antivirus project.

Resources:

ASCII Table / Extended ASCII Codes- A table with ASCII and hex character codes.

NickCiske.com | Tools- Tools for converting various number systems (including hex) to text.

Online Hex Dump- An online hex dumper (if the title wasn’t obvious).

Python Programming Language — Official Site- Home to the famous scripting/programming language. It’s easy to learn, and allows you to make quick little functions that save you a ton of time. Python can also make robust applications. That’s why we’ll be using it to make the antivirus program in the next part of the tutorial.

Viruslist.com -Information About Viruses, Hackers, and Spam- A huge virus info site.

Virus Detection Signatures- Free virus signatures. There’s probably better stuff out there though.

That’s all for part one folks! If you love, hate, or don’t care, leave a comment. There might be a small wait for part two, but it should be within the next week or two (I might do some smaller posts for a while, and I’ll be out of a town later this week). Thanks for reading!

-Vainentree