Dear linus.walleij,
I am sorry to trouble you. Would you mind give a hand?
I got a job that should log the RAM memory access in the QEMU. First, I should find out the code line in QEMU to trap all RAM memory access. After some efforts, I have some conclusions:
1. I have found the function dealing with the translation from the virtual address to physical address in the guest of QEMU, such as [target-arm/helper.c:get_phys_addr], and the page fault handler function [target-arm/helper.c:cpu_arm_handle_mmu_fault]. However, we have not found out the routine of accessing physical address from MMU TLB entry. 2. We have understood the relation of the guest physical address and host virtual address, and found the function [exec.c:qemu_get_ram_ptr] translating the guest physical address to host virtual address.
I am pressed for time. Would you mind to help me to check it? If you know the location of the code lines or related messages, please tell me. Thank you for your kindness. The QEMU I used getting from [* http://lists.linaro.org/pipermail/linaro-announce/2012-February/000095.html* ].
Thanks, Jerry Zhou
On 14 March 2012 14:01, 周春华 uulinux@gmail.com wrote:
I got a job that should log the RAM memory access in the QEMU. First, I should find out the code line in QEMU to trap all RAM memory access. After some efforts, I have some conclusions:
- I have found the function dealing with the translation from the virtual
address to physical address in the guest of QEMU, such as [target-arm/helper.c:get_phys_addr], and the page fault handler function [target-arm/helper.c:cpu_arm_handle_mmu_fault]. However, we have not found out the routine of accessing physical address from MMU TLB entry.
The TLB lookup and reading of the host RAM is done by generated code (for instance on an x86 host this code is generated by tcg_out_qemu_ld() and tcg_out_qemu_st() in tcg/i386/tcg-target.c).
NB that the QEMU TLB goes straight from guest virtual address to host address when it's reading RAM, without passing through a guest physical address.
I'm afraid there's no convenient place to put logging of memory accesses in this code, because it is designed for speed rather than ease of instrumentation.
-- PMM
Dear Peter Maydell,
Thanks for your reply. Would you mind give me more help?
My PC architecture is x86, so the tcg_out_qemu_ld() and tcg_out_qemu_st() is in tcg/i386/tcg-target.c. Nevertheless, it is difficult for me to understand them completely.
Do you means that the QEMU TLB maps the guest virtual address to host virtual address, and the begging and end virtual addresses of the memory allocated for RAM device emulating are the RAM physical begging and end address from guest view? If so, it seems hard to monitor the guest physical memory.
However, [exec.c:qemu_get_ram_ptr] seems to get a host virtual address from a guest physical address. It confuses me.
Would you mind explain more details about QEMU TLB?
Any comments will be appreciated. Thanks much!
Thanks, Jerry
2012/3/14 Peter Maydell peter.maydell@linaro.org
On 14 March 2012 14:01, 周春华 uulinux@gmail.com wrote:
I got a job that should log the RAM memory access in the QEMU. First, I should find out the code line in QEMU to trap all RAM memory access.
After
some efforts, I have some conclusions:
- I have found the function dealing with the translation from the
virtual
address to physical address in the guest of QEMU, such as [target-arm/helper.c:get_phys_addr], and the page fault handler function [target-arm/helper.c:cpu_arm_handle_mmu_fault]. However, we have not
found
out the routine of accessing physical address from MMU TLB entry.
The TLB lookup and reading of the host RAM is done by generated code (for instance on an x86 host this code is generated by tcg_out_qemu_ld() and tcg_out_qemu_st() in tcg/i386/tcg-target.c).
NB that the QEMU TLB goes straight from guest virtual address to host address when it's reading RAM, without passing through a guest physical address.
I'm afraid there's no convenient place to put logging of memory accesses in this code, because it is designed for speed rather than ease of instrumentation.
-- PMM
2012/3/15 周春华 uulinux@gmail.com:
Do you means that the QEMU TLB maps the guest virtual address to host virtual address,
Yes.
and the begging and end virtual addresses of the memory allocated for RAM device emulating are the RAM physical begging and end address from guest view?
I don't know what you mean by this. RAM in qemu need not be contiguous in guest physical address space, and it need not be contiguous in host physical address space either.
If so, it seems hard to monitor the guest physical memory.
Yes, that's what I said.
However, [exec.c:qemu_get_ram_ptr] seems to get a host virtual address from a guest physical address. It confuses me.
Obviously QEMU knows how to map between guest physical addresses and host virtual addresses, or it wouldn't work. This function is one very small part of a complicated subsystem which caches the "guest virtual -> guest physical -> host virtual" lookups so we don't need to do them again and again when we execute load or store instructions.
If you want to follow the code in more detail, when QEMU gets a "TLB miss" (ie it doesn't know where the RAM for a guest virtual address is) it calls target-arm/helper.c:cpu_arm_handle_mmu_fault(). This calls get_phys_addr() to do a page table walk and convert the guest virtual address to a guest physical address. Assuming that succeeded, it calls exec.c:tlb_set_page(), passing the guest virtual and guest physical addresses, to add a TLB entry. This function calls memory_region_get_ram_ptr() which in turn calls qemu_get_ram_ptr(), getting the host virtual address. We can then cache the host virtual address for this guest virtual address in the TLB entry. Later on when we actually execute a guest load or store instruction we will pull the TLB entry out of the data structure and use the host virtual address cached in it. tcg_out_qemu_ld/st are the functions which generate the native code which gets the TLB entry and loads via the cached host virtual address.
Note that there are other slow paths for memory access which don't use the TLB and instead do go via physical addresses at the time they need to do the load/store.
-- PMM
Dear Peter Maydell,
I am very appreciated for you great help. There is still a question exist:
Note that there are other slow paths for memory access which don't
use the TLB and instead do go via physical addresses at the time they need to do the load/store.
I want to know how to use these slow paths. Will they will "guest virtual -> guest physical -> host virtual" again and again when we execute load or store instructions. If so, I prefer to use them instead. It will help me to log the memory access.
Any way, I want to thank you again.
Best Regards, Jerry
2012/3/15 Peter Maydell peter.maydell@linaro.org
2012/3/15 周春华 uulinux@gmail.com:
Do you means that the QEMU TLB maps the guest virtual address to host virtual address,
Yes.
and the begging and end virtual addresses of the memory allocated for RAM device emulating are the RAM physical begging and end address from guest view?
I don't know what you mean by this. RAM in qemu need not be contiguous in guest physical address space, and it need not be contiguous in host physical address space either.
If so, it seems hard to monitor the guest physical memory.
Yes, that's what I said.
However, [exec.c:qemu_get_ram_ptr] seems to get a host virtual address
from
a guest physical address. It confuses me.
Obviously QEMU knows how to map between guest physical addresses and host virtual addresses, or it wouldn't work. This function is one very small part of a complicated subsystem which caches the "guest virtual -> guest physical -> host virtual" lookups so we don't need to do them again and again when we execute load or store instructions.
If you want to follow the code in more detail, when QEMU gets a "TLB miss" (ie it doesn't know where the RAM for a guest virtual address is) it calls target-arm/helper.c:cpu_arm_handle_mmu_fault(). This calls get_phys_addr() to do a page table walk and convert the guest virtual address to a guest physical address. Assuming that succeeded, it calls exec.c:tlb_set_page(), passing the guest virtual and guest physical addresses, to add a TLB entry. This function calls memory_region_get_ram_ptr() which in turn calls qemu_get_ram_ptr(), getting the host virtual address. We can then cache the host virtual address for this guest virtual address in the TLB entry. Later on when we actually execute a guest load or store instruction we will pull the TLB entry out of the data structure and use the host virtual address cached in it. tcg_out_qemu_ld/st are the functions which generate the native code which gets the TLB entry and loads via the cached host virtual address.
Note that there are other slow paths for memory access which don't use the TLB and instead do go via physical addresses at the time they need to do the load/store.
-- PMM
2012/3/15 周春华 uulinux@gmail.com:
Peter Maydell wrote:
Note that there are other slow paths for memory access which don't use the TLB and instead do go via physical addresses at the time they need to do the load/store.
I want to know how to use these slow paths. Will they will "guest virtual -> guest physical -> host virtual" again and again when we execute load or store instructions.
No, you don't get to choose whether slow or fast paths are used. QEMU uses the fast paths where it can, and occasionally also the slow paths in odd corner cases or less often used operations.
-- PMM
Dear Peter Maydell,
Thank you for your help. I got an idea to log the guest physical address, would you mind help me to check it?
Because the function tcg_out_qemu_ld() and tcg_out_qemu_st() know the guest virtual address should be accessed, we can call the [target-arm/helper.c:get_phys_addr] function to translate the guest virtual address to the guest physical address and log the guest physical address.
Is it feasible? I know it will be slow the QEMU.
Best Regards, Jerry
2012/3/15 Peter Maydell peter.maydell@linaro.org
2012/3/15 周春华 uulinux@gmail.com:
Peter Maydell wrote:
Note that there are other slow paths for memory access which don't use the TLB and instead do go via physical addresses at the time they need to do the load/store.
I want to know how to use these slow paths. Will they will "guest
virtual ->
guest physical -> host virtual" again and again when we execute load or store instructions.
No, you don't get to choose whether slow or fast paths are used. QEMU uses the fast paths where it can, and occasionally also the slow paths in odd corner cases or less often used operations.
-- PMM
2012/3/15 周春华 uulinux@gmail.com:
Dear Peter Maydell,
Thank you for your help. I got an idea to log the guest physical address, would you mind help me to check it?
Because the function tcg_out_qemu_ld() and tcg_out_qemu_st() know the guest virtual address should be accessed, we can call the [target-arm/helper.c:get_phys_addr] function to translate the guest virtual address to the guest physical address and log the guest physical address.
Is it feasible? I know it will be slow the QEMU.
Depends what you mean by 'feasible'. It's not totally impossible. Bear in mind that you'll have to write C code which writes out x86 instructions which do the actual function call, and that you will need to do it without accidentally trashing any registers which are in use at that point. You'll also need to find all the slow path accesses which don't go through this point.
You might find it simpler to attack the problem at a higher level by modifying the translator to output calls to tracing helper functions before every load/store instruction.
This is all getting pretty complicated, though, and I wouldn't recommend it without a decent understanding of how QEMU works...
-- PMM
Hi Perter,
I read some code lines of QEMU in the past week and have more understanding about QEMU. But there are also a lot of things unclear in my mind. And I beg you help me again.
First, I found there are some x86 instructions translated from TCG instructions will access the "RAM device" directly. Do you think so? Is it fast path you said? And the slow path means it's necessary to call these functions in softmmu_template.h (such as glue(glue(__ld, SUFFIX), MMUSUFFIX)) to get the host virtual address from guest virtual address. If the access the "RAM device" directly is true, I think it is very difficult to monitor. Do you have a good idea?
Second, about the following suggestion:
You might find it simpler to attack the problem at a higher level
by modifying the translator to output calls to tracing helper functions before every load/store instruction.
Did it means add an new TCG instructions to tracing it? The new TCG instruction will be generated when disassemble ARM LD/ST instructions, and then the new TCG instruction will generate the call of tracing helper function when they are translated to X86. In the tracing helper function, the RAM access log will out put. Does these your means?
Thanks, Jerry
2012/3/15 Peter Maydell peter.maydell@linaro.org
2012/3/15 周春华 uulinux@gmail.com:
Dear Peter Maydell,
Thank you for your help. I got an idea to log the guest physical address, would you mind help me to check it?
Because the function tcg_out_qemu_ld() and tcg_out_qemu_st() know the
guest
virtual address should be accessed, we can call the [target-arm/helper.c:get_phys_addr] function to translate the guest
virtual
address to the guest physical address and log the guest physical address.
Is it feasible? I know it will be slow the QEMU.
Depends what you mean by 'feasible'. It's not totally impossible. Bear in mind that you'll have to write C code which writes out x86 instructions which do the actual function call, and that you will need to do it without accidentally trashing any registers which are in use at that point. You'll also need to find all the slow path accesses which don't go through this point.
You might find it simpler to attack the problem at a higher level by modifying the translator to output calls to tracing helper functions before every load/store instruction.
This is all getting pretty complicated, though, and I wouldn't recommend it without a decent understanding of how QEMU works...
-- PMM
-
Jerry -
Peter Maydell -
周春华