;未完待续;持续更新中;

目录

序言

0 存储设备

0.1 机械盘

0.2 SSD

0.3 NVDIMM

1 VFS

1.0 基础组件

1.0.1 iov_iter

1.0.2 iomap

1.1 inode

1.1.1 inode的生命周期

1.2 Dentry

1.2.1 dentry的功能

1.2.2 dentry操作

1.2.3 root dentry

1.2.4 d_splice_alias

1.2.5 lookup

1.2.6 symbol link

1.3 mount

1.3.1 嫁接

1.3.2 loop mount

1.4 page cache

1.4.1 radix tree

1.4.2 page cache tags

1.4.3 mmap

1.4.4 direct IO

1.5 DAX

1.5.1 Pmem

1.5.2 fs/dax

序言

Linux的存储栈包括;

• API;将POSIX系统调用接口引入VFS层
• VFS;维护page cache、dcache、inode cache等结构的组织和内容;同时向下对接文件系统
• FS;包括各种文件系统;ext2、ext4、xfs、btrfs等
• Logical-Block;包括基于device-mapper的lvm、md-raid、bcache、multi-path等
• Block;即通常我们所讲的块层;向下对接存储协议;抽象存储设备的物理特性;包括最大IO size、是否支持cache等;同时提供各种IO调度(电梯算法可以认为是默认)策略
• 设备驱动;比如nvme和scsi类的存储设备;
• 物理存储设备

0 存储设备

存储栈的组件在设计时;一个重要的考量因素;就是如何更好的使用存储设备;使其发挥出最好的性能;所以;在研究Linux存储栈之前;我们先要了解下;各种存储设备的特点。

0.1 机械盘

我们评估价下磁盘以下性能指标;
Host;Intel(R) Xeon(R) Gold 5218 CPU ; 2.30GHz
HDD;ST2000NM0055-1V4;磁盘容量1.8T

顺序IO;不同IOsize (--bs)

fio --bs=256K --size=512G  --rw=read --norandommap --direct=1 --ioengine=libaio --iodepth=1 --runtime=60 --name=test --filename=/dev/sdb

随机IO;不同IO Range ;--size;

fio --bs=512 --size=1536G  --rw=randread --norandommap --direct=1 --ioengine=libaio --iodepth=1 --runtime=60 --name=test --filename=/dev/sdb --group_reporting

通过以上数据;我们可以大致估算;磁盘执行IO主要分成三部分的时间;

• 软件栈开销;包括host和firmware;参考512字节顺序IO;应该在24 us以下;
• 磁头从磁盘上拷贝数据;大致200M/s;这与测试机械盘的理论带宽上限接近;
• 磁头移动;在移动范围超过128M之后;磁头移动时间就已经达到3ms;超过1G之后达到5ms;

所以;这块机械盘的瓶颈有个决定因素;

• 200M/s的带宽上限;这个通常只有在测试中使用纯顺序IO才能达到。
• 磁头移动范围;在实践中;这才是真正决定磁盘性能的因素;

还是以这个磁盘为例;如果是xfs;默认4个Allocation Group;每个AG的大小512G(最后一个不足512G);不同目录的文件;通常会被分配到不同的AG;当多个目录的文件同时IO时;其范围很可能在256G以上;也就是磁头移动造成的时间大概在7ms;那么;每秒处理的IO数;也就是IOPS最高只能在143;这就是此时盘的瓶颈;此时;IOsize越大;盘的带宽就会越高。

0.2 SSD

;占坑;

0.3 NVDIMM

NVDIMM这个名字中包含了该种类型设备的两个特性;

• Non-Volatile;非易失性;即掉电不丢失;
• DIMM;这是该种设备的接口形式;跟DDR一样;这意味着;它有非常高的带宽上限;并且可以通过load/store指令访问;

即NVDIMM指的是一种使用DIMM接口的非易失性存储设备;

        Intel近些年推出的Optane DC PMM就是其中的一种;其基于3D XPoint存储技术;同样的存储技术还基于NVMe接口实现了Optane SSD;其实我们也可以给硬盘实现DIMM接口;只不过成本和性能就看不过去了。

        Optane DCPMM第一代产品代号为Apache Pass;也就是我们常说的AEP;其第二代产品代号为Barlow Pass;不过;网传英特尔将终止基于 3D XPoint 技术的 Optane 业务。

 NVDIMM的性能数据来自网络上的两篇Paper《Basic Performance Measurements of the Intel Optane DC Persistent Memory Module》《Initial Experience with 3D XPoint Main Memory》;感兴趣的同学可以到https://download.csdn.net/download/home19900111/86736994https://download.csdn.net/download/home19900111/86736994

Initial Experience with 3D XPoint Main Memoryhttps://download.csdn.net/download/home19900111/86737051

 图中的LDRAM的意思是本地内存访问;RDRAM跨numa内存访问;

L stands for regular loads,

LN stands for non-temporal loads,

SF stands for stores followed by clflush

SN stands for non- temporal stores

SO stands for stores followed by clflushopt

SW stands for stores followed by clwb.

 从图中的数据来看;虽然Opane的数据与真正的内存相比依然有些差距;但是已经基本在一个数量级;结合《Initial Experience with 3D XPoint Main Memory》中的结论;

• The memory access performance of 3D XPoint is mod- estly lower than that of DRAM. Compared to DRAM, 3D XPoint’s read performance can be up to 2.5x slower, and its write performance can be up to 2.3x slower.

• The write performance of 3D XPoint is quite similar to its read performance. Writes are at most 2.3x slower than reads. The asymmetric effect of NVM reads and writes are modest.

• Persisting writes to 3D XPoint incur drastic cost for sequential and random writes. The slowdowns can be at least 10.7x and 2.9x for sequential writes and random writes, respectively.

• The performance of 3D XPoint degrades as an application accesses more data. We suspect that techniques such as caching are exploited in the 3D Xpoint modules to improve performance. This effect is less significant when the size of data visited is 32GB or larger for the 256GB 3D XPoint module.

与HDD或者Nand SSD对比;这无疑是;史诗级;的性能提升;但是;在实际使用中;我们发现AEP最大的问题是;由于其使用load/store指令直接访问;无法实现异步;这造成相关应用的CPU利用率显著升高。

Intel后来推出了基于FPGA的硬件加速模块;使用内部的几百上千的并发单元访问AEP。

不过;现阶段AEP仍然无法真正在引发计算系统革命;当NVDIMM的性能达到DDR且成本可控时;才会真正颠覆现有的架构。

关于nvdimm的持久化编程;参考掉电保护域;

Asynchronous DRAM Refresh (ADR) is a feature supported on Intel products which flushes the write-protected data buffers and places the DRAM in self-refresh. This process is critical during a power loss event or system crash to ensure the data is in a safe and consistent state on persistent memory. By default, ADR does not flush the processor caches. A platform that supports ADR only includes persistent memory and the memory controller’s write pending queues within the persistence domain. This is the reason data in the CPU caches must be flushed by the application using the CLWB, CLFLUSHOPT, CLFLUSH, non-temporal stores, or WBINVD machine instructions.

Enhanced Asynchronous DRAM Refresh (eADR) requires that a non-maskable interrupt (NMI) routine be called to flush the CPU caches before the ADR event can begin. Applications running on an eADR platform do not need to perform flush operations because the hardware should flush the data automatically, but they are still required to perform an SFENCE operation to maintain write order correctness. Stores should be considered persistent only when they are globally visible, which the SFENCE guarantees.

实现持久化编程的几个关键指令;

• CLFLUSH;This instruction, supported in many generations of CPU, flushes a single cache line. Historically, this instruction is serialized, causing multiple CLFLUSH instructions to execute one after the other, without any concurrency.
• CLFLUSHOPT    This instruction, newly introduced for persistent memory support, is like CLFLUSH but without the serialization. To flush a range, software executes a CLFLUSHOPT instruction for each 64-byte cache line in the range, followed by a single SFENCE instruction to ensure the flushes are complete before continuing.  CLFLUSHOPT is optimized (hence the name) to allow some concurrency when executing multiple CLFLUSHOPT instructions back-to-back
• CLWB    Another newly introduced instruction, CLWB stands for cache line write back. The effect is the same as CLFLUSHOPT except that the cache line may remain valid in the cache (but no longer dirty, since it was flushed). This makes it more likely to get a cache hit on this line as the data is accessed again later
• NT stores     Another feature that has been around for a while in x86 CPUs is the non-temporal  store. These stores are “write combining” and bypass the CPU cache, so using them does not require a flush.
• WBINVD;This kernel-mode-only instruction flushes and invalidates every cache line on the CPU that executes it. After executing this on all CPUs, all stores to persistent memory are certainly in the persistence domain, but all cache lines are empty, impacting  performance. In addition, the overhead of sending a message to each CPU to execute this instruction can be significant. Because of this, WBINVD is only expected to be used by the kernel for flushing very large ranges, many megabytes at least.

1 VFS

1.0 基础组件

1.0.1 iov_iter

参考

What is iov_iterhttps://jianchwa.github.io/fs/xfs_v2.html#iov_iter

1.0.2 iomap

我们首先看下iomap结构

#define IOMAP_HOLE	0	/* no blocks allocated, need allocation */
#define IOMAP_DELALLOC	1	/* delayed allocation blocks */
#define IOMAP_MAPPED	2	/* blocks allocated at ;addr */
#define IOMAP_UNWRITTEN	3	/* blocks allocated at ;addr in unwritten state */
#define IOMAP_INLINE	4	/* data inline in the inode */

struct iomap {
	u64			addr;   /* disk offset of mapping, bytes */
	loff_t		offset;	/* file offset of mapping, bytes */
	u64			length;	/* length of mapping, bytes */
	u16			type;	/* type of mapping */
	u16			flags;	/* flags for mapping */
	struct block_device	*bdev;	/* block device for I/O */
    ...
	const struct iomap_page_ops *page_ops;
};

dax_iomap_rw()
---
    while ((ret = iomap_iter(&iomi, ops)) > 0)
                  ---
                  (1) iomap_end
                  (2) advance iter by iter.processed
                  (3) iomap_begin, based on iter.inode, iter.pos, iter.len, get iomap.addr, iomap.offset, iomap.length
                  ---
        iomi.processed = dax_iomap_iter(&iomi, iter);
                         ---
                         read/write on iomap.addr ; (iomap.offset - iter.pos)
                         ---
---

ext2_iget()
---
	inode = iget_locked(sb, ino);
	if (!(inode->i_state & I_NEW))
		return inode;

	ei = EXT2_I(inode);
	ei->i_block_alloc_info = NULL;

    //这个步骤需要跟存储设备做IO
	raw_inode = ext2_get_inode(inode->i_sb, ino, &bh);
	// Fill the in-core inode from on-disk inode
	unlock_new_inode(inode);
---

iget_locked()
---
	spin_lock(&inode_hash_lock);
	inode = find_inode_fast(sb, head, ino);
	spin_unlock(&inode_hash_lock);
	if (inode) {
		...
        //在此处等待I_NEW被清除掉
		wait_on_inode(inode);
		return inode;
	}

	inode = alloc_inode(sb);
	if (inode) {
		struct inode *old;

		spin_lock(&inode_hash_lock);
		/* We released the lock, so.. */
		old = find_inode_fast(sb, head, ino);
		if (!old) {
			inode->i_ino = ino;
			spin_lock(&inode->i_lock);
			inode->i_state = I_NEW;
			hlist_add_head(&inode->i_hash, head);
			spin_unlock(&inode->i_lock);
			inode_sb_list_add(inode);
			spin_unlock(&inode_hash_lock);

			/* Return the locked inode with I_NEW set, the
			 * caller is responsible for filling in the contents
			 */
			return inode;
		}
		...
	}
---

iput_final()
---
	if (op->drop_inode)
		drop = op->drop_inode(inode);
	else
		drop = generic_drop_inode(inode);
		// !inode->i_nlink || inode_unhashed(inode);

    //引用计数为0之后;将inode放到lru中;等待回收
	if (!drop && (sb->s_flags & SB_ACTIVE)) {
		inode_add_lru(inode);
		spin_unlock(&inode->i_lock);
		return;
	}
	...
   //对于已经被文件系统除名的文件;即i_nlink = 0, 不必再保留inode;直接释放掉
	inode->i_state |= I_FREEING;
	if (!list_empty(&inode->i_lru))
		inode_lru_list_del(inode);
	spin_unlock(&inode->i_lock);

	evict(inode);
}
---

super_cache_scan()
  -> prune_icache_sb()
    -> inode_lru_isolate()
	-> dispose_list()

follow_managed()
---
			struct vfsmount *mounted = lookup_mnt(path);
			if (mounted) {
				dput(path->dentry);
				if (need_mntput)
					mntput(path->mnt);
				path->mnt = mounted;
				path->dentry = dget(mounted->mnt_root);
				need_mntput = true;
				continue;
			}
---

struct mount *__lookup_mnt(struct vfsmount *mnt, struct dentry *dentry)
{
	struct hlist_head *head = m_hash(mnt, dentry);
	struct mount *p;

	hlist_for_each_entry_rcu(p, head, mnt_hash)
		if (&p->mnt_parent->mnt == mnt && p->mnt_mountpoint == dentry)
			return p;
	return NULL;
}

do_new_mount()
  -> vfs_kern_mount()
    -> do_add_mount()
	  -> graft_tree()
	    -> attach_recursive_mnt()
	      -> commit_tree()
            -> __attach_mnt()

static void __attach_mnt(struct mount *mnt, struct mount *parent)
{
	hlist_add_head_rcu(&mnt->mnt_hash,
			   m_hash(&parent->mnt, mnt->mnt_mountpoint));
	list_add_tail(&mnt->mnt_child, &parent->mnt_mounts);
}

struct dentry *d_splice_alias(struct inode *inode, struct dentry *dentry)
 {
-       struct dentry *new = NULL;
-
        if (IS_ERR(inode))
                return ERR_CAST(inode);
 
-       if (inode && S_ISDIR(inode->i_mode)) {
-               spin_lock(&inode->i_lock);
-               new = __d_find_any_alias(inode);
-               if (new) {
-                       if (!IS_ROOT(new)) {
-                               spin_unlock(&inode->i_lock);
-                               dput(new);
-                               iput(inode);
-                               return ERR_PTR(-EIO);
-                       }
                        // 如果dentry位于new树下;这会造成loop
-                       if (d_ancestor(new, dentry)) {
-                               spin_unlock(&inode->i_lock);
-                               dput(new);
-                               iput(inode);
-                               return ERR_PTR(-EIO);
-                       }
-                       write_seqlock(&rename_lock);
-                       __d_move(new, dentry, false);
-                       write_sequnlock(&rename_lock);
-                       spin_unlock(&inode->i_lock);
-                       security_d_instantiate(new, inode);
-                       iput(inode);
-               } else {
-                       /* already taking inode->i_lock, so d_add() by hand */
-                       __d_instantiate(dentry, inode);
-                       spin_unlock(&inode->i_lock);
-                       security_d_instantiate(dentry, inode);
-                       d_rehash(dentry);
-               }
-       } else {
-               d_instantiate(dentry, inode);
-               if (d_unhashed(dentry))
-                       d_rehash(dentry);
-       }
-       return new;
-}
-EXPORT_SYMBOL(d_splice_alias);

1. lookup ;/;
2. lookup ;home;         in dentry of ;/;
3. lookup ;will;         in dentry of ;/home;
4. lookup ;Desktop;      in dentry of ;/home/will;
5. lookup ;linux-stable; in dentry of ;/home/will/Desktop;

The cases where rcu-walk cannot continue are:
* NULL dentry (ie. creat or negative lookup)
* parent with ->d_op->d_hash
* parent with d_inode->i_op->permission or ACLs
* Following links
* Dentry with ->d_op->d_revalidate

typedef struct seqcount {
	unsigned sequence;
} seqcount_t;

static inline unsigned __read_seqcount_begin(const seqcount_t *s)
{
	unsigned ret;

repeat:
	ret = READ_ONCE(s->sequence);
	if (unlikely(ret & 1)) {
		cpu_relax();
		goto repeat;
	}
	return ret;
}

static inline void raw_write_seqcount_begin(seqcount_t *s)
{
	s->sequence;;;
	smp_wmb();
}

static inline void raw_write_seqcount_end(seqcount_t *s)
{
	smp_wmb();
	s->sequence;;;
}

以下代码已经将部分错误分支处理去掉;以使代码变得更清晰
link_path_walk()
---
	while (*name==;/;)
		name;;;
	if (!*name)
		return 0;

	for(;;) {
        //对路径中 .. 和 . 的处理;类似于/home/will/Deskop/../Download
		type = LAST_NORM;
		if (name[0] == ;.;) switch (hashlen_len(hash_len)) {
			case 2:
				if (name[1] == ;.;) {
					type = LAST_DOTDOT;
					nd->flags |= LOOKUP_JUMPED;
				}
				break;
			case 1:
				type = LAST_DOT;
		}

        // hash_name函数会以;/;作为name的结尾来返回len
		hash_len = hash_name(nd->path.dentry, name);

		nd->last.hash_len = hash_len;
		nd->last.name = name;
		nd->last_type = type;

        // 指向路径的下一个元素
		name ;= hashlen_len(hash_len);
		if (!*name)
			goto OK;

        // 路径中加几个;/;都没有关系
		do {
			name;;;
		} while (unlikely(*name == ;/;));

		if (unlikely(!*name)) {
		} else {
            //处理当前的路径元素
			/* not the last component */
			err = walk_component(nd, WALK_FOLLOW | WALK_MORE);
		}
		...
	}
}

/home/will/Desktop/sourcecode/linux-stable
                        |
                        ;--> /mnt/disk1/AAAA/sourcecode
                                          |
                                          ;--> /mnt/disk2/BBBB

路径的解析过程为


stack[0]   home -> will -> Dektop -> sourcecode                   ;-> linux-stable
                            ____________/__________              |
                           /                                     |
stack[1]                    mnt  ->  disk1   -> AAAA              |
                                     ____________/__________     |
                                    /                            |
                                     mnt  ->  disk2   -> BBBB    -;


与symlink stack有关的代码为;

扩展stack;


push stack;

walk_component() -> step_into() -> pick_link() -> nd_alloc_stack()

link_path_walk()
---
		if (err) {
            // 此处调用的是文件系统的get_link方法
			const char *s = get_link(nd);
            ...
			if (unlikely(!s)) {
			} else {
				nd->stack[nd->depth - 1].name = name;
				name = s;
				continue;
			}
		}
---

pop stack:
link_path_walk()
---
		if (unlikely(!*name)) {
OK:
			/* pathname body, done */
			if (!nd->depth)
				return 0;

            // Stack is popped here
			name = nd->stack[nd->depth - 1].name;
			/* trailing symlink, done */
			if (!name)
				return 0;
			/* last component of nested symlink */
			err = walk_component(nd, WALK_FOLLOW);
		} else {
			/* not the last component */
			err = walk_component(nd, WALK_FOLLOW | WALK_MORE);
		}
---
NO WALK_MORE
walk_component() -> step_into() -> put_link() -> nd->depth--

struct mount *__lookup_mnt(struct vfsmount *mnt, struct dentry *dentry)
{
	struct hlist_head *head = m_hash(mnt, dentry);
	struct mount *p;

	hlist_for_each_entry_rcu(p, head, mnt_hash)
		if (&p->mnt_parent->mnt == mnt && p->mnt_mountpoint == dentry)
			return p;
	return NULL;
}

do_add_mount()
---
	parent = real_mount(path->mnt);
	err = graft_tree(newmnt, parent, mp);
	  -> attach_recursive_mnt()
	    -> mnt_set_mountpoint()
---

walk_component()
  -> follow_managed()
	---
		if (managed & DCACHE_MOUNTED) {
			struct vfsmount *mounted = lookup_mnt(path);
			if (mounted) {
				dput(path->dentry);
				if (need_mntput)
					mntput(path->mnt);
				path->mnt = mounted;
				path->dentry = dget(mounted->mnt_root);
				need_mntput = true;
				continue;
			}
	---
  -> step_into()
    -> path_to_nameidata()
	   ---
			nd->path.mnt = path->mnt;
			nd->path.dentry = path->dentry;
	   ---

修改task_struct.fs.root
ksys_chroot()
---
	error = user_path_at(AT_FDCWD, filename, lookup_flags, &path);
	set_fs_root(current->fs, &path);
	---
	mntget(path->mnt);
	dget(path->dentry);
	spin_lock(&fs->lock);
	write_seqcount_begin(&fs->seq);
	old_root = fs->root;
	fs->root = *path;
	write_seqcount_end(&fs->seq);
	spin_unlock(&fs->lock);
	if (old_root.dentry) {
		dput(path->dentry);
		mntput(path->mnt);
	}
	---
---

这个结构会在fork的时候;被子任务继承
copy_process()
  -> copy_fs()
    -> copy_fs_struct()
	   ---
		spin_lock(&old->lock);
		fs->root = old->root;
		path_get(&fs->root);
		fs->pwd = old->pwd;
		path_get(&fs->pwd);
		spin_unlock(&old->lock);
	   ---

do_loopback()
---
	// bind mount directory
	old = real_mount(old_path.mnt);
	// mount point
	parent = real_mount(path->mnt);

	mnt = clone_mnt(old, old_path.dentry, 0);
	---
		mnt = alloc_vfsmnt(old->mnt_devname);
		...
		//增加bind mount directory所在文件系统的sb的引用计数
		atomic_inc(&sb->s_active);
		mnt->mnt.mnt_sb = sb;
        // bind mount的root就是bind mount directory
		mnt->mnt.mnt_root = dget(root);
		//这个会在set_mount_point的时候再次更新
		mnt->mnt_mountpoint = mnt->mnt.mnt_root;
		mnt->mnt_parent = mnt;
		lock_mount_hash();
		list_add_tail(&mnt->mnt_instance, &sb->s_mounts);
		unlock_mount_hash();
	---
	err = graft_tree(mnt, parent, mp);
	  -> attach_recursive_mnt()
	    -> mnt_set_mountpoint(dest_mnt, dest_mp, source_mnt);

mount -t bind /home/will/Desktop/sourcecode /mnt/readonly
mount -o remount,ro /mnt/readonly

filename_create()
do_renameat2()
do_unlinkat()
do_last()
....
mnt_want_write()
  -> __mnt_want_write()
   ---
  	preempt_disable();
	mnt_inc_writers(mnt);
	...
	smp_rmb();
	if (mnt_is_readonly(m)) {
		mnt_dec_writers(mnt);
		ret = -EROFS;
	}
	preempt_enable();

	return ret;
   ---

__radix_tree_create()
---
    //遍历的停止条件;比较order
	while (shift > order) {
		shift -= RADIX_TREE_MAP_SHIFT;
		if (child == NULL) {
			/* Have to add a child node.  */
			child = radix_tree_node_alloc(gfp, node, root, shift,
							offset, 0, 0);
			rcu_assign_pointer(*slot, node_to_entry(child));
			if (node)
				node->count;;;
		} else if (!radix_tree_is_internal_node(child))
			break;

		/* Go a level down */
		node = entry_to_node(child);
		offset = radix_tree_descend(node, &child, index);
		slot = &node->slots[offset];
	}
---


insert_entries()
---
	if (node) {
		if (order > node->shift)
			n = 1 << (order - node->shift);
		else
			n = 1;
		offset = get_slot_offset(node, slot);
	} else {
		n = 1;
		offset = 0;
	}

	if (n > 1) {
		offset = offset & ~(n - 1);
		slot = &node->slots[offset];
	}
	sibling = xa_mk_sibling(offset);
	...
	for (i = 0; i < n; i;;) {
		struct radix_tree_node *old = rcu_dereference_raw(slot[i]);
        // 第一个成员保存item;其他保存sibling指针
		if (i) {
			rcu_assign_pointer(slot[i], sibling);
		} else {
			rcu_assign_pointer(slot[i], item);
		}
        //释放掉所有子成员
		if (xa_is_node(old))
			radix_tree_free_nodes(old);
		if (xa_is_value(old))
			node->nr_values--;
	}
---

handle_pte_fault()
---
	// vmf->pte is not NULL and present
	if (vmf->flags & FAULT_FLAG_WRITE) {
		if (!pte_write(entry))
			return do_wp_page(vmf);
		entry = pte_mkdirty(entry);
	}
---
do_wp_page()会调用到page_mkwrite;其中会调用set_page_dirty()和
file_update_time()等;

而write_cache_pages()则会调用
clear_page_dirty_for_io() -> page_mkclean()清除相关标记位

pmem_submit_bio()
---
	bio_for_each_segment(bvec, bio, iter) {
		if (op_is_write(bio_op(bio)))
			rc = pmem_do_write(pmem, bvec.bv_page, bvec.bv_offset,
				iter.bi_sector, bvec.bv_len);
		else
			rc = pmem_do_read(pmem, bvec.bv_page, bvec.bv_offset,
				iter.bi_sector, bvec.bv_len);
		if (rc) {
			bio->bi_status = rc;
			break;
		}
	}
	bio_endio(bio);
---

pmem_submit_bio()
  -> pmem_do_write()
     //数据的写入地址;从sector转换而来
	-> phys_addr_t pmem_off = sector * 512 ; pmem->data_offset;
	-> write_pmem();
	  -> memcpy_flushcache()
	    -> __memcpy_flushcache()
      //写入size一定是512对齐的
		---
		/* 4x8 movnti loop */
		while (size >= 32) {
			asm(;movq    (%0), %%r8
;
		    	;movq   8(%0), %%r9
;
		    	;movq  16(%0), %%r10
;
		    	;movq  24(%0), %%r11
;
		    	;movnti  %%r8,   (%1)
;
		    	;movnti  %%r9,  8(%1)
;
		    	;movnti %%r10, 16(%1)
;
		    	;movnti %%r11, 24(%1)
;
		    	:: ;r; (source), ;r; (dest)
		    	: ;memory;, ;r8;, ;r9;, ;r10;, ;r11;);
			dest ;= 32;
			source ;= 32;
			size -= 32;
		}
		---

dax_iomap_rw()
---
    while ((ret = iomap_iter(&iomi, ops)) > 0)
                  ---
                  (1) iomap_end
                  (2) advance iter by iter.processed
                  (3) iomap_begin, based on iter.inode, iter.pos, iter.len, get iomap.addr, iomap.offset, iomap.length
                  ---
        iomi.processed = dax_iomap_iter(&iomi, iter);
                         ---
                         read/write on iomap.addr ; (iomap.offset - iter.pos)
                         ---
---
dax_iomap_iter()
---
	/*
	             iter->pos     iter->len
	                |            |
	     |--------------------------------|
	            mmapped chunk in file
	 iomap.offset                       iomap.length
	 */
	length = min(iter->len, iter->iomap.offset ; iter->iomap.length - iter->pos);
	pos = iter->pos;
    end = pos ; length;

	while (pos < end) {
		// Copy one page per round
		unsigned offset = pos & (PAGE_SIZE - 1);
		const size_t size = ALIGN(length ; offset, PAGE_SIZE);
		const sector_t sector = (iomap->addr ; (pos & PAGE_MASK) - iomap->offset) >> 9;

		// Get the address of pmem for direct access
		ret = bdev_dax_pgoff(bdev, sector, size, &pgoff);
		map_len = dax_direct_access(dax_dev, pgoff, PHYS_PFN(size),
				&kaddr, NULL);
		...
		if (iov_iter_rw(iter) == WRITE)
			xfer = dax_copy_from_iter(dax_dev, pgoff, kaddr,
					map_len, iter);
		else
			xfer = dax_copy_to_iter(dax_dev, pgoff, kaddr,
					map_len, iter);

		pos ;= xfer;
		length -= xfer;
		...
	}

---

write_ext_stat()
  -> compute_ext_disk_stats()
	 ---
		xds->util  = S_VALUE(sdp->tot_ticks, sdc->tot_ticks, itv);
	 ---

read_sysfs_dlist_part_stat()
  -> read_sysfs_file_stat()
	---
		i = fscanf(fp, %lu %lu %lu %lu %lu %lu %lu %u %u %u %u;,
			   &rd_ios, &rd_merges_or_rd_sec, &rd_sec_or_wr_ios, &rd_ticks_or_wr_sec,
			   &wr_ios, &wr_merges, &wr_sec, &wr_ticks, &ios_pgr, &tot_ticks, &rq_ticks);
	---
filename = /sys/block/xxx/stat

void update_io_ticks(struct hd_struct *part, unsigned long now)
{
	unsigned long stamp;
again:
	stamp = READ_ONCE(part->stamp);
	if (unlikely(stamp != now)) {
		if (likely(cmpxchg(&part->stamp, stamp, now) == stamp)) {
			__part_stat_add(part, io_ticks, 1);
		}
	}
	if (part->partno) {
		part = &part_to_disk(part)->part0;
		goto again;
	}
}

blk_mq_bio_to_request()
  -> blk_account_io_start()
	-> update_io_ticks()

__blk_mq_end_request()
  -> blk_account_io_done()
	-> update_io_ticks(

commit 5b18b5a737600fd20ba2045f320d5926ebbf341a
Author: Mikulas Patocka <mpatocka;redhat.com>
Date:   Thu Dec 6 11:41:19 2018 -0500

    block: delete part_round_stats and switch to less precise counting
    
    We want to convert to per-cpu in_flight counters.
    
    The function part_round_stats needs the in_flight counter every jiffy, it
    would be too costly to sum all the percpu variables every jiffy, so it
    must be deleted. part_round_stats is used to calculate two counters -
    time_in_queue and io_ticks.
    
    time_in_queue can be calculated without part_round_stats, by adding the
    duration of the I/O when the I/O ends (the value is almost as exact as the
    previously calculated value, except that time for in-progress I/Os is not
    counted).
    
    io_ticks can be approximated by increasing the value when I/O is started
    or ended and the jiffies value has changed. If the I/Os take less than a
    jiffy, the value is as exact as the previously calculated value. If the
    I/Os take more than a jiffy, io_ticks can drift behind the previously
    calculated value.
    
    Signed-off-by: Mikulas Patocka <mpatocka;redhat.com>
    Signed-off-by: Mike Snitzer <snitzer;redhat.com>
    Signed-off-by: Jens Axboe <axboe;kernel.dk>

static void part_round_stats_single(struct request_queue *q,
                                   struct hd_struct *part, unsigned long now,
                                   unsigned int inflight)
{
       if (inflight) {
               __part_stat_add(part, time_in_queue,
                               inflight * (now - part->stamp));
               __part_stat_add(part, io_ticks, (now - part->stamp));
       }
       part->stamp = now;
}