Virtual Memory

[Lateral:Brain-Q]

Hi,

There can be a large number of processes each with its own 2 GB of private virtual address space.

Courtesy: http://support.microsoft.com/?kbid=555223


In 32-bit Windows OS's, Virtual Memory Addresses range from 0 to 2^32 - 1 and if each memory cell can store 1Byte the total amount of data that could be stored is 4GB.


Now, if the Total Virtual Address Space = 4GB, how is it then possible to have a large number of Processes each with its own 2GB of Private Virtual Address Space as mentioned above from the Microsoft article?

Also, how are the Virtual Memory Addresses mapped to the Physical Memory Addresses?

Thanks

[nihil]

Hmmm,

Now, if the Total Virtual Address Space = 4GB, how is it then possible to have a large number of Processes each with its own 2GB of Private Virtual Address Space as mentioned above from the Microsoft article?

You can have as many as you like, you just cannot run them all at the same time because you are faced with an architectural limit.

Incidentally, I believe that it is possible to tweak the system to exceed 2GB for an individual process, but the limit of 4GB remains.

In 32-bit Windows OS's, Virtual Memory Addresses range from 0 to 2^32 - 1 and if each memory cell can store 1Byte the total amount of data that could be stored is 4GB.

In theory, yes, but be careful. Windows 9x/ME systems will not do this. They were supposed to be able to support up to 2GB of RAM, but MS never managed to get it to work. It is interesting that they released 4 operating systems over 5~6 years, all with this same problem.

Also, how are the Virtual Memory Addresses mapped to the Physical Memory Addresses?

This is done in the hardware.

[Lateral:Brain-Q]

nihil, Thankyou for your explanation. If you will, I would like to ask a question about Page Fault.

From the definition of what I've read of Page Fault, a Page Fault or Invalid Page Fault error occurs when the OS cannot find the data in virtual memory . This usually happens when the virtual memory area or the table that maps virtual memory addresses to physical memory addresses becomes corrupt.

I can understand the explanation of Page Fault, when the Table that maps the Virtual memory addresses to Physical memory addresses gets corrupt.

My question is,

Is data stored in Virtual Memory?

The reason I ask this question is, take for example a Process is running in the Shared 2GB of Virtual Address Space. My understanding of Virtual Memory is, a Process is given a Pool of Virtual Addresses that maps to Physical Addresses, which is where the data is actually stored. If it is, how can data be stored at a virtual address? Or have I completely misunderstood the concept of Virtual Memory?

Thanks

[cacosapo]

From the definition of what I've read of Page Fault, a Page Fault or Invalid Page Fault error occurs when the OS cannot find the data in virtual memory . This usually happens when the virtual memory area or the table that maps virtual memory addresses to physical memory addresses becomes corrupt.

I can understand the explanation of Page Fault, when the Table that maps the Virtual memory addresses to Physical memory addresses gets corrupt.

Page Fault is not a Page Table corruption error; instead is the way that hardware supports Virtual Memory.
Page Fault means: "Hello Operating System. I've tried to translate this virtual address to a reall address but i couldnt find it on Page Table. So I've interrupted the process and call you to fix it"

Just That. Windows (as so many other OSes) implements Multiple Private Address Spaces, so each process owns a "private" virtual storage that is not shared with other processes. Each process has a private address space. But Since you can have 4GB (32 bits) of private address space, your process usually dont use all that most of time. So Operating system (that build and maintain Page Tables) build an incomplete page table just mapping pages that you are current use (or those that are current on Real Memory). Hardware uses those tables to map virtual address to real addresses. When hardware unit (MMU) doesnt find a match on the table, it causes a Page Fault interrupt just to inform O.S. to fix it. (Or end the process if it is a invalid address)

My question is,
Is data stored in Virtual Memory?
The reason I ask this question is, take for example a Process is running in the Shared 2GB of Virtual Address Space. My understanding of Virtual Memory is, a Process is given a Pool of Virtual Addresses that maps to Physical Addresses, which is where the data is actually stored. If it is, how can data be stored at a virtual address? Or have I completely misunderstood the concept of Virtual Memory?

Everything needed to run a instruction is on Real storage - data (operands) and the binany code itself. But since the program is running in "Virtual storage mode", it only "sees" virtual addresses. So all data and code are contained in virtual storage. Also, some pieces of program+data can be moved to disk (page file, swap, etc) when its not being used.


Although 32 bits can give you 4GB of addressing, Windows (and other OSes) usually occupy a piece of virtual address space for its code. This is necessary because a lot of operations are done thru O.S. routines and some data exchange between processes is done also thru O.S. common area. In Windows Case, it reserves 2GB for OS and 2GB for problem program.
You can expand it for 3GB using /3GB switch on boot.

[Lateral:Brain-Q]

Originally posted here by cacosapo
Everything needed to run a instruction is on Real storage - data (operands) and the binany code itself. But since the program is running in "Virtual storage mode", it only "sees" virtual addresses. So all data and code are contained in virtual storage.

cacosapo, Thankyou for your explanation: Now, from where is the 4GB of Memory being stolen from, to realize Virtual Memory?

In the explanation given above, as per the first statement, data and binary code itself is on real storage and also the last but one statement says, data and code are contained in virtual storage . Which is correct?

[nihil]

Ah! I think that I see your problem.

The MMU (memory management utility) handles both physical and virtual memory by means of addresses.

The physical memory (RAM) is what actually does the "work" so to speak, whilst virtual memory is a storage area awaiting to be allocated physical memory and processed.

Virtual memory resides on your hard drive. Both contain information.

[cacosapo]

Lets say that:
Real storage exists but programs can't see it and
Virtual storage doesnt exist but programs can address it

Real memory is a hardware; everything is currently used by CPU must reside on it. It includes code + data.

Virtual Memory is a software concept that is supported by a special unit (MMU). So since all (problem) programs only "see" virtual memory -- all code and data (from its point of view) is contained on that storage. However its is a "simulation"; every address of virtual memory must be mapped to an address of real memory to be executed by CPU. MMU is in charge to do the necessary conversion between virtual-to-real addresses. Since not all addres space storage is need at the same time, O.S. uses to preserve real memory usage and transfer unused parts of every virtual storage to disk.
Its easy to get: if you have a PC with 256MB of RAM (Real memory), could you run a program that use 300 MB of memory? The answer is YES. Sometimes you can have performance issues but...

Virtual Memory is given by O.S. to programs as Politics words; they deliver what they dont have

[Lateral:Brain-Q]

cacosapo & nihil, Thankyou for the explanations. Greatly appreciated.

If I can continue the subject of Virtual Memory, on Intel based Processors, the Page Size for Virtual Memory is either Small [4KB] or Large [4MB].

My question is,

Would there be any improved performance say if one uses 4MB Page instead of 4KB?

Thanks

/Edit: Here is a scenario:

Say, the system is set to use Large Pages [4MB]. Since Memory protection is on a per-Page basis.

Size of Ntoskrnl.exe on my Windows XP SP2 = 2129KB
Size of Hal.dll = 129KB
Memory Left On the 4MB Page = 1838KB

If somehow read/write code also exist on the same Page that has the read-only Operating System code, will the attributes of the Page change to read/write?

If it does, would it be possible to alter an instruction [say Ntoskrnl.exe] at a particular Memory address or alter the data of the Operating System code?

Am I just dreaming or is this something possible?

Thanks

[nihil]

Ah! page size?

If you have a page size of 4, and a file of 4.2, it will use two pages to make a total of 8. Now, if you have a pge size of 400, it will use only one page, but you will waste one hell of a lot of space?

[Lateral:Brain-Q]

Originally posted here by nihil
Ah! page size?

If you have a page size of 4, and a file of 4.2, it will use two pages to make a total of 8. Now, if you have a pge size of 400, it will use only one page, but you will waste one hell of a lot of space?

Sorry nihil. I'm not sure if I'm able to decipher the above statement. Is the statement referring to 4KB Clusters on the Hard Disk?

I should have been more clear. My previous post was referring to Page Sizes for Virtual Memory. From what I understand, a Virtual Page can be 4KB [Small Page] or 4MB [2MB on Physical Address Extension]. On my machine under System Properties it says, Physical Address Extension and I'm guessing I can set the Virtual Page Size on my machine to 2MB even if the Cluster Size on the Hard Disk is 4KB. Would that be an accurate statement?

Thanks