网站空间报价单,做网站龙岗,做逆战网站的名字吗,网站404页面源码适用场景
看到B站技术部门的文章#xff0c;是关于如何在k8s集群部署Elastic Search和Click House等IO密集型数据库应用的。 因为要充分利用NvME SSD盘的IOPS#xff0c;所有数据库应用都直接调用本地SSD盘做为stateful application的 Persistent Volumes。为了可用动态的分…适用场景
看到B站技术部门的文章是关于如何在k8s集群部署Elastic Search和Click House等IO密集型数据库应用的。 因为要充分利用NvME SSD盘的IOPS所有数据库应用都直接调用本地SSD盘做为stateful application的 Persistent Volumes。为了可用动态的分配大小不一的本地磁盘空间给k8s创建的pods使用最合适的本地磁盘系统应该为LVM。 最近两天我在自己的虚拟集群里实现了lvm类型的storage class并测试了调用此storagecalss的PVC 确认可用正常生成需要的PV并bound给到pods使用。
再说一次LVM 类型的StorageClass 适用于有状态的应用直接调用本地磁盘来确保高IOPS的使用场景如数据库应用
知识储备建议
本文将使用到如下知识建议阅读者有如下知识储备
LVM相关概念如 PV VG LV fdisk命令的使用CSI相关概念Helmchart/releasek8s存储相关PV PVC SCStorageClass
实现原理
Worker node安装lvm并创建相应vgMaster CP安装CSI插件并生成相应StorageClass。 当API server 检测到PVC时自动通过StorageClass生成lvm的PV。 原谅我懒得画个图
步骤分解并注释
Worker node 实现lvm存储
给虚拟机添加新磁盘
因为我的集群是个vmware workstaion的集群首先需要给worker node的虚拟机增加第二块磁盘然后
rootslave2:/home/bill# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 19G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 975M 0 part
sr0 11:0 1 631M 0 rom
# lsblk是list block设备的命令发现还是只有一个名叫 sda 的磁盘 并没有找到第二块新增的盘。我新增的SCSI格式磁盘理论上不需要重启即可挂载
rootslave2:/home/bill# echo - - - | sudo tee /sys/class/scsi_host/host*/scan
- - -
# Chatgpt告诉我这个命令可用强制刷新
rootslave2:/home/bill# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 19G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 975M 0 part
sdb 8:16 0 20G 0 disk
sr0 11:0 1 631M 0 rom
# 好了有了第二块 sdb 磁盘了
rootslave2:/home/bill# sudo fdisk /dev/sdb
# 自从上云后很少管理裸金属服务器了很多命令如fdisk iptables 很久不用要生疏了。 fdisk是linux常用的用来对磁盘进行分区管理并格式化的工具。
Welcome to fdisk (util-linux 2.38.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.Device does not contain a recognized partition table.
Created a new DOS (MBR) disklabel with disk identifier 0x907350bf.Command (m for help): h
h: unknown command
# 别学我习惯用h 人家分明说”m for help“
# 你知道什么是partition 么 这个是要进行fdisk操作的最基础概念需要去google下如果你不知道的话。
Command (m for help): mHelp:DOS (MBR)a toggle a bootable flagb edit nested BSD disklabelc toggle the dos compatibility flagGenericd delete a partition #慎用F list free unpartitioned spacel list known partition typesn add a new partition #新增分区必用p print the partition table #常用用来显示当前磁盘和分区情况t change a partition typev verify the partition tablei print information about a partitionMiscm print this menuu change display/entry unitsx extra functionality (experts only)ScriptI load disk layout from sfdisk script fileO dump disk layout to sfdisk script fileSave Exitw write table to disk and exit #修改了磁盘分区partition后一定要用w只有这样才可用保存你的变更q quit without saving changes # 最后使用Create a new labelg create a new empty GPT partition tableG create a new empty SGI (IRIX) partition tableo create a new empty MBR (DOS) partition tables create a new empty Sun partition tableCommand (m for help): p #打印看下fdisk是否检测到了新增磁盘Disk /dev/sdb: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: VMware Virtual S
Units: sectors of 1 * 512 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x907350bf
#检测到/dev/sdb了你最好有扇区这么个概念不过SSD盘已经没有扇区了fdisk延用了而已。
Command (m for help): n
Partition typep primary (0 primary, 0 extended, 4 free)e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-41943039, default 2048):
Last sector, /-sectors or /-size{K,M,G,T,P} (2048-41943039, default 41943039): Created a new partition 1 of type Linux and of size 20 GiB.Command (m for help): p
Disk /dev/sdb: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: VMware Virtual S
Units: sectors of 1 * 512 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x907350bfDevice Boot Start End Sectors Size Id Type
/dev/sdb1 2048 41943039 41940992 20G 83 LinuxCommand (m for help): mHelp:DOS (MBR)a toggle a bootable flagb edit nested BSD disklabelc toggle the dos compatibility flagGenericd delete a partitionF list free unpartitioned spacel list known partition typesn add a new partitionp print the partition tablet change a partition typev verify the partition tablei print information about a partitionMiscm print this menuu change display/entry unitsx extra functionality (experts only)ScriptI load disk layout from sfdisk script fileO dump disk layout to sfdisk script fileSave Exitw write table to disk and exitq quit without saving changesCreate a new labelg create a new empty GPT partition tableG create a new empty SGI (IRIX) partition tableo create a new empty MBR (DOS) partition tables create a new empty Sun partition tableCommand (m for help): d
Selected partition 1
Partition 1 has been deleted.Command (m for help): p
Disk /dev/sdb: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: VMware Virtual S
Units: sectors of 1 * 512 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x907350bfCommand (m for help): n # n 用来新增一个分区partition因为这时候磁盘还是裸的即没有分区更没有对分区进行格式化如xfsext4以便系统可用拿来存放数据文件
Partition typep primary (0 primary, 0 extended, 4 free)e extended (container for logical partitions)
Select (default p): p # n 命令下的子选项 p表示primary partition主分区而不是e extended 扩展分区
Partition number (1-4, default 1): #你就空着吧太细节了 不用管
First sector (2048-41943039, default 2048): #你就空着吧太细节了 不用管
Last sector, /-sectors or /-size{K,M,G,T,P} (2048-41943039, default 41943039): #你就空着吧太细节了 不用管这个就是扇区了其实可用精细的调整一些物理磁盘的东西的不过我们的主要目标是k8s管理lvm本地磁盘所以都略了。Created a new partition 1 of type Linux and of size 20 GiB.
# 给创建了一个type 为 Linux 的分区
Command (m for help): t #t 我们需要的是lvm type的分区所以用 t 来trans
Selected partition 1 #只有1个partition /dev/sdb1
Hex code or alias (type L to list all): L # 我们来用L看一下有多少分区类型可选00 Empty 27 Hidden NTFS Win 82 Linux swap / So c1 DRDOS/sec (FAT-
01 FAT12 39 Plan 9 83 Linux c4 DRDOS/sec (FAT-
02 XENIX root 3c PartitionMagic 84 OS/2 hidden or c6 DRDOS/sec (FAT-
03 XENIX usr 40 Venix 80286 85 Linux extended c7 Syrinx
04 FAT16 32M 41 PPC PReP Boot 86 NTFS volume set da Non-FS data
05 Extended 42 SFS 87 NTFS volume set db CP/M / CTOS / .
06 FAT16 4d QNX4.x 88 Linux plaintext de Dell Utility
07 HPFS/NTFS/exFAT 4e QNX4.x 2nd part 8e Linux LVM df BootIt
08 AIX 4f QNX4.x 3rd part 93 Amoeba e1 DOS access
09 AIX bootable 50 OnTrack DM 94 Amoeba BBT e3 DOS R/O
0a OS/2 Boot Manag 51 OnTrack DM6 Aux 9f BSD/OS e4 SpeedStor
0b W95 FAT32 52 CP/M a0 IBM Thinkpad hi ea Linux extended
0c W95 FAT32 (LBA) 53 OnTrack DM6 Aux a5 FreeBSD eb BeOS fs
0e W95 FAT16 (LBA) 54 OnTrackDM6 a6 OpenBSD ee GPT
0f W95 Extd (LBA) 55 EZ-Drive a7 NeXTSTEP ef EFI (FAT-12/16/
10 OPUS 56 Golden Bow a8 Darwin UFS f0 Linux/PA-RISC b
11 Hidden FAT12 5c Priam Edisk a9 NetBSD f1 SpeedStor
12 Compaq diagnost 61 SpeedStor ab Darwin boot f4 SpeedStor
14 Hidden FAT16 3 63 GNU HURD or Sys af HFS / HFS f2 DOS secondary
16 Hidden FAT16 64 Novell Netware b7 BSDI fs f8 EBBR protective
17 Hidden HPFS/NTF 65 Novell Netware b8 BSDI swap fb VMware VMFS
18 AST SmartSleep 70 DiskSecure Mult bb Boot Wizard hid fc VMware VMKCORE
1b Hidden W95 FAT3 75 PC/IX bc Acronis FAT32 L fd Linux raid auto
1c Hidden W95 FAT3 80 Old Minix be Solaris boot fe LANstep
1e Hidden W95 FAT1 81 Minix / old Lin bf Solaris ff BBT
24 NEC DOS Aliases:linux - 83swap - 82extended - 05uefi - EFraid - FDlvm - 8Elinuxex - 85
Hex code or alias (type L to list all): 8e
Changed type of partition Linux to Linux LVM.
# 我们需要change为代码是 8e16进制数字Linux LVM格式
Command (m for help): p
Disk /dev/sdb: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: VMware Virtual S
Units: sectors of 1 * 512 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x907350bfDevice Boot Start End Sectors Size Id Type
/dev/sdb1 2048 41943039 41940992 20G 8e Linux LVM
# 再打印一边看看 看最后一行 /dev/sdb1 这个确实是Linux LVM 了
Command (m for help): w # 千万别忘了保存变更
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
接下来创建lvm下的pv vg不需要创建lv并格式化
rootslave2:/home/bill# apt install lvm2 #安装lvm2
rootslave2:/home/bill# sudo pvcreate /dev/sdb1 # 创建pvPhysical volume /dev/sdb1 successfully created.
rootslave2:/home/bill# sudo pvsPV VG Fmt Attr PSize PFree /dev/sdb1 lvm2 --- 20.00g 20.00g
rootslave2:/home/bill# sudo pvdisplay #显示创建好的pv/dev/sdb1 is a new physical volume of 20.00 GiB--- NEW Physical volume ---PV Name /dev/sdb1VG Name PV Size 20.00 GiBAllocatable NOPE Size 0 Total PE 0Free PE 0Allocated PE 0PV UUID 8YdNRX-I9ZR-lXKc-8yvp-OMvN-eZWm-dDecux #PV的uuidrootslave2:/home/bill# sudo vgcreate vg-k8s-test /dev/sdb1 Volume group vg-k8s-test successfully created#为整个磁盘创建为了一个名为 vg-k8s-test 的volume group#附加思考可用将2个物理磁盘创建为一个volume group么 可用将lv创建在两个不同磁盘的vg上么能或者不能各有什么利弊
rootslave2:/home/bill# sudo vgsVG #PV #LV #SN Attr VSize VFree vg-k8s-test 1 0 0 wz--n- 20.00g 20.00g# vgs命令可看名为 vg-k8s-test的vg已经存在了记住整个名字后面要用到
rootslave2:/home/bill# sudo vgdisplay--- Volume group ---VG Name vg-k8s-testSystem ID Format lvm2Metadata Areas 1Metadata Sequence No 1VG Access read/writeVG Status resizable #当然是resizable的了 这才是lvm的具体优势所在MAX LV 0Cur LV 0Open LV 0Max PV 0Cur PV 1Act PV 1VG Size 20.00 GiBPE Size 4.00 MiBTotal PE 5119Alloc PE / Size 0 / 0 Free PE / Size 5119 / 20.00 GiBVG UUID GWJesr-2Qa8-UMya-4UIF-kkWG-qGGj-Lfe0gt #VG也有自己的uuid
也不好少了 明天将重头戏 也就是k8s如何实现lvm类型的StorageClass讲完。
