왜이 메모리 먹는 사람이 실제로 메모리를 먹지 않습니까?

왜이 메모리 먹는 사람이 실제로 메모리를 먹지 않습니까?

---

Unix 서버에서 메모리 부족 (OOM) 상황을 시뮬레이션하는 프로그램을 만들고 싶습니다. 나는이 매우 간단한 메모리 먹는 사람을 만들었습니다 :

#include <stdio.h>
#include <stdlib.h>

unsigned long long memory_to_eat = 1024 * 50000;
size_t eaten_memory = 0;
void *memory = NULL;

int eat_kilobyte()
{
    memory = realloc(memory, (eaten_memory * 1024) + 1024);
    if (memory == NULL)
    {
        // realloc failed here - we probably can't allocate more memory for whatever reason
        return 1;
    }
    else
    {
        eaten_memory++;
        return 0;
    }
}

int main(int argc, char **argv)
{
    printf("I will try to eat %i kb of ram\n", memory_to_eat);
    int megabyte = 0;
    while (memory_to_eat > 0)
    {
        memory_to_eat--;
        if (eat_kilobyte())
        {
            printf("Failed to allocate more memory! Stucked at %i kb :(\n", eaten_memory);
            return 200;
        }
        if (megabyte++ >= 1024)
        {
            printf("Eaten 1 MB of ram\n");
            megabyte = 0;
        }
    }
    printf("Successfully eaten requested memory!\n");
    free(memory);
    return 0;
}

memory_to_eat정확히 50GB의 RAM 인 정의 된만큼 많은 메모리를 사용 합니다. 메모리를 1MB 씩 할당하고 더 많은 할당에 실패한 지점을 정확하게 인쇄하여 어느 최대 값을 먹을 수 있는지 알 수 있습니다.

문제는 작동한다는 것입니다. 1GB의 실제 메모리가있는 시스템에서도.

맨 위로 확인하면 프로세스가 50GB의 가상 메모리와 1MB 미만의 상주 메모리 만 사용한다는 것을 알 수 있습니다. 실제로 그것을 소비하는 메모리 이터를 만드는 방법이 있습니까?

시스템 사양 : 스왑없이 가상화 된 오버 커밋이 활성화 된 Linux 커널 3.16 ( Debian )

---

귀하의 경우 malloc()구현 (AN 통해 시스템 커널에서 메모리를 요청 sbrk()또는 mmap()시스템 호출), 커널은 당신이 메모리를 요청하고 그것이 당신의 주소 공간 내에 배치 될 위치하는 메모를합니다. 실제로 해당 페이지를 매핑하지는 않습니다 .

When the process subsequently accesses memory within the new region, the hardware recognizes a segmentation fault and alerts the kernel to the condition. The kernel then looks up the page in its own data structures, and finds that you should have a zero page there, so it maps in a zero page (possibly first evicting a page from page-cache) and returns from the interrupt. Your process does not realize that any of this happened, the kernels operation is perfectly transparent (except for the short delay while the kernel does its work).

This optimization allows the system call to return very quickly, and, most importantly, it avoids any resources to be committed to your process when the mapping is made. This allows processes to reserve rather large buffers that they never need under normal circumstances, without fear of gobbling up too much memory.

---

So, if you want to program a memory eater, you absolutely have to actually do something with the memory you allocate. For this, you only need to add a single line to your code:

int eat_kilobyte()
{
    if (memory == NULL)
        memory = malloc(1024);
    else
        memory = realloc(memory, (eaten_memory * 1024) + 1024);
    if (memory == NULL)
    {
        return 1;
    }
    else
    {
        //Force the kernel to map the containing memory page.
        ((char*)memory)[1024*eaten_memory] = 42;

        eaten_memory++;
        return 0;
    }
}

Note that it is perfectly sufficient to write to a single byte within each page (which contains 4096 bytes on X86). That's because all memory allocation from the kernel to a process is done at memory page granularity, which is, in turn, because of the hardware that does not allow paging at smaller granularities.

---

All the virtual pages start out copy-on-write mapped to the same zeroed physical page. To use up physical pages, you can dirty them by writing something to each virtual page.

If running as root, you can use mlock(2) or mlockall(2) to have the kernel wire up the pages when they're allocated, without having to dirty them. (normal non-root users have a ulimit -l of only 64kiB.)

As many others suggested, it seems that the Linux kernel doesn't really allocate the memory unless you write to it

An improved version of the code, which does what the OP was wanting:

This also fixes the printf format string mismatches with the types of memory_to_eat and eaten_memory, using %zi to print size_t integers. The memory size to eat, in kiB, can optionally be specified as a command line arg.

The messy design using global variables, and growing by 1k instead of 4k pages, is unchanged.

#include <stdio.h>
#include <stdlib.h>

size_t memory_to_eat = 1024 * 50000;
size_t eaten_memory = 0;
char *memory = NULL;

void write_kilobyte(char *pointer, size_t offset)
{
    int size = 0;
    while (size < 1024)
    {   // writing one byte per page is enough, this is overkill
        pointer[offset + (size_t) size++] = 1;
    }
}

int eat_kilobyte()
{
    if (memory == NULL)
    {
        memory = malloc(1024);
    } else
    {
        memory = realloc(memory, (eaten_memory * 1024) + 1024);
    }
    if (memory == NULL)
    {
        return 1;
    }
    else
    {
        write_kilobyte(memory, eaten_memory * 1024);
        eaten_memory++;
        return 0;
    }
}

int main(int argc, char **argv)
{
    if (argc >= 2)
        memory_to_eat = atoll(argv[1]);

    printf("I will try to eat %zi kb of ram\n", memory_to_eat);
    int megabyte = 0;
    int megabytes = 0;
    while (memory_to_eat-- > 0)
    {
        if (eat_kilobyte())
        {
            printf("Failed to allocate more memory at %zi kb :(\n", eaten_memory);
            return 200;
        }
        if (megabyte++ >= 1024)
        {
            megabytes++;
            printf("Eaten %i  MB of ram\n", megabytes);
            megabyte = 0;
        }
    }
    printf("Successfully eaten requested memory!\n");
    free(memory);
    return 0;
}

---

A sensible optimisation is being made here. The runtime does not actually acquire the memory until you use it.

A simple memcpy will be sufficient to circumvent this optimisation. (You might find that calloc still optimises out the memory allocation until the point of use.)

---

Not sure about this one but the only explanation that I can things of is that linux is a copy-on-write operating system. When one calls fork the both processes point to the same physically memory. The memory is only copied once one process actually WRITES to the memory.

I think here, the actual physical memory is only allocated when one tries to write something to it. Calling sbrk or mmap may well only update the kernel's memory book-keep. The actual RAM may only be allocated when we actually try to access the memory.

참고URL : https://stackoverflow.com/questions/33234139/why-doesnt-this-memory-eater-really-eat-memory