面向异构内存的可靠管理关键技术研究
查看服务器拓扑架构
hwloc
lstopo
León E A, Goglin B, Proaño A R. M&MMs: navigating complex memory spaces with hwloc[C]//Proceedings of the International Symposium on Memory Systems. 2019: 149-155.
表现异构内存性能信息
启用ACPI中定义的HMAT
https://lwn.net/Articles/727348/
NUMA Balancing
ACPI
# 依赖安装
sudo apt-get install git build-essential gdb-multiarch qemu-system-misc gcc-riscv64-linux-gnu binutils-riscv64-linux-gnu gcc-riscv64-unknown-elf binutils-riscv64-unknown-elf
参考链接： https://pdos.csail.mit.edu/6.S081/2021/tools.html
git clone git://g.csail.mit.edu/xv6-labs-2020
cd xv6-labs-2020
git checkout util
make qemu
启用修改后内核，并拆分NUMA节点
修改代码，编译内核
- 解压kernel-openEuler-22.03-LTS代码
- 修改代码
- cp -v /boot/config-$(uname -r) .config config-直接按TAB自动补全
- make ARCH=arm64 -j60
- make modules_install -j60
- make install ARCH=arm64
- reboot
编译内核后，需要重新拆分NUMA node
iasl -sa SRAT.dsl
iasl -sa HMAT.dsl
iasl -sa SLIT.dsl
cp SRAT.aml HMAT.aml SLIT.aml /home/Lihoo/kernel/firmware/acpi/
cd ..
# 将修改的SRAT之类的保存到virtual_node_initramfs_2209
find kernel | cpio -c -o > /boot/virtual_node_initramfs_2209
# 将原生的initramfs附加上
cat /boot/initramfs-$(uname -r).img >> /boot/virtual_node_initramfs_2209
vim /boot/efi/EFI/openEuler/grub.cfg
# 把initrd那行改为：
# initrd /virtual_node_initramfs_2209
reboot
简化步骤，为上面两步之和
make ARCH=arm64 -j60
make modules_install -j60
make install ARCH=arm64
find kernel | cpio -c -o > /boot/virtual_node_initramfs_2209
cat /boot/initramfs-$(uname -r).img >> /boot/virtual_node_initramfs_2209
vim /boot/efi/EFI/openEuler/grub.cfg
# 指定相应内核版本的initrd为virtual_node_initramfs_2209
# initrd为linux内核加载初始的ramdisk
cat /boot/efi/EFI/openEuler/grub.cfg |grep "menuentry "
grub2-mkconfig -o /boot/efi/EFI/openEuler/grub.cfg
reboot
新增CONFIG项后重新编译内核
make mrproper
cp -v /boot/config-$(uname -r) .config
make menuconfig
make -j4
make modules_install -j4
make install
reboot
错误注入（UCE）
获取/proc/<pid>/maps下指定区域的一段虚拟地址作为注入UCE的虚拟地址段，转化为物理地址
部分虚拟地址段不能注入UCE，如：共享库和text段
启用日志 & 调试
cat /dev/null > /var/log/messages
echo 1 > /proc/sys/kernel/numa_balancing_reliability_debug
echo 1 > /proc/sys/kernel/numa_balancing_reliability_page_alloc
cat /var/log/messages
# core dump配置
ulimit -a
# core dump目录
cd /var/lib/systemd/coredump/
# core dump文件解压
lz4 -d core.XSBench.0.89c225516dca4878b8e88278b63d3b30.596845.1682218282000000.lz4
dmesg | grep <pid>
dmesg -w
系统信息查看
# 查看当前config
zcat /proc/config.gz > /home/Lihoo/running.config
# 查看cache回写策略：
dmidecode -t cache
# 查看物理CPU个数
cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l
# 查看每个物理CPU中core的个数(即核数)
cat /proc/cpuinfo| grep "cpu cores"| uniq
# 查看逻辑CPU的个数
cat /proc/cpuinfo| grep "processor"| wc -l
KVM虚拟机磁盘扩容
cd /home/lihaoDong/image2
qemu-img resize openEuler.qcow2 +10G
fdisk -l //看到有多余的空间
参考链接 操作2 新增/dev/vda#
fdisk /dev/vda
t ->4 ->L ->30 ->w /dev/vda6走到这一步，/boot是part？不是lvm？
fdisk -l //查看新增的/dev/vda# 类型是否为Linux LVM
mkfs.ext4 /dev/vda4
pvcreate /dev/vda4
pvdisplay
vgextend openeuler /dev/vda4
pvdisplay
lvextend -L +13G /dev/mapper/openeuler-root
lvs
resize2fs /dev/mapper/openeuler-root
df -h
https://blog.csdn.net/weixin_40436144/article/details/86235432
透明大页（THP）
#查看大页内存使用情况
grep Huge /proc/meminfo
#查看各个numa节点的大页内存情况
cat /sys/devices/system/node/node0/meminfo | fgrep Huge
#查看大页内存挂载情况
cat /proc/mounts
#查看透明大页启动情况
cat /sys/kernel/mm/transparent_hugepage/enabled
#启动透明大页
echo always > /sys/kernel/mm/transparent_hugepage/enabled
#禁用透明大页
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo "vm.nr_hugepages=200" >> /etc/sysctl.conf
https://www.cnblogs.com/halberd-lee/p/12802918.html
KVM虚拟机
# 虚拟机配置
vim openEulerHMA.xml
virsh define openEulerHMA.xml
virsh start openEulerHMA
virsh start openEuler
virt-manager
virsh shutdown openEuler
virsh restart openEuler
其他参考链接：
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-working_with_the_grub_2_boot_loader#sec-Making_Persistent_Changes_to_a_GRUB_2_Menu_Using_the_grubby_Tool
https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/05_ACPI_Software_Programming_Model/ACPI_Software_Programming_Model.html?highlight=hmat