io-mapping: Provide iomap_local variant

Similar to kmap local provide a iomap local variant which only disables
migration, but neither disables pagefaults nor preemption.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20201118204007.561220818@linutronix.de

Documentation/driver-api/io-mapping.rst

@@ -20,55 +20,71 @@ A mapping object is created during driver initialization using::
 mappable, while 'size' indicates how large a mapping region to
 enable. Both are in bytes.
 
-This _wc variant provides a mapping which may only be used
+This _wc variant provides a mapping which may only be used with
-with the io_mapping_map_atomic_wc or io_mapping_map_wc.
+io_mapping_map_atomic_wc(), io_mapping_map_local_wc() or
+io_mapping_map_wc().
 
-With this mapping object, individual pages can be mapped either atomically
+With this mapping object, individual pages can be mapped either temporarily
-or not, depending on the necessary scheduling environment. Of course, atomic
+or long term, depending on the requirements. Of course, temporary maps are
-maps are more efficient::
+more efficient. They come in two flavours::
+
+	void *io_mapping_map_local_wc(struct io_mapping *mapping,
+				      unsigned long offset)
 
 	void *io_mapping_map_atomic_wc(struct io_mapping *mapping,
 				       unsigned long offset)
 
-'offset' is the offset within the defined mapping region.
+'offset' is the offset within the defined mapping region.  Accessing
-Accessing addresses beyond the region specified in the
+addresses beyond the region specified in the creation function yields
-creation function yields undefined results. Using an offset
+undefined results. Using an offset which is not page aligned yields an
-which is not page aligned yields an undefined result. The
+undefined result. The return value points to a single page in CPU address
-return value points to a single page in CPU address space.
+space.
 
-This _wc variant returns a write-combining map to the
+This _wc variant returns a write-combining map to the page and may only be
-page and may only be used with mappings created by
+used with mappings created by io_mapping_create_wc()
-io_mapping_create_wc
 
-Note that the task may not sleep while holding this page
+Temporary mappings are only valid in the context of the caller. The mapping
-mapped.
+is not guaranteed to be globaly visible.
 
-::
+io_mapping_map_local_wc() has a side effect on X86 32bit as it disables
+migration to make the mapping code work. No caller can rely on this side
+effect.
 
+io_mapping_map_atomic_wc() has the side effect of disabling preemption and
+pagefaults. Don't use in new code. Use io_mapping_map_local_wc() instead.
+
+Nested mappings need to be undone in reverse order because the mapping
+code uses a stack for keeping track of them::
+
+ addr1 = io_mapping_map_local_wc(map1, offset1);
+ addr2 = io_mapping_map_local_wc(map2, offset2);
+ ...
+ io_mapping_unmap_local(addr2);
+ io_mapping_unmap_local(addr1);
+
+The mappings are released with::
+
+	void io_mapping_unmap_local(void *vaddr)
 	void io_mapping_unmap_atomic(void *vaddr)
 
-'vaddr' must be the value returned by the last
+'vaddr' must be the value returned by the last io_mapping_map_local_wc() or
-io_mapping_map_atomic_wc call. This unmaps the specified
+io_mapping_map_atomic_wc() call. This unmaps the specified mapping and
-page and allows the task to sleep once again.
+undoes the side effects of the mapping functions.
 
-If you need to sleep while holding the lock, you can use the non-atomic
+If you need to sleep while holding a mapping, you can use the regular
-variant, although they may be significantly slower.
+variant, although this may be significantly slower::
-
-::
 
 	void *io_mapping_map_wc(struct io_mapping *mapping,
 				unsigned long offset)
 
-This works like io_mapping_map_atomic_wc except it allows
+This works like io_mapping_map_atomic/local_wc() except it has no side
-the task to sleep while holding the page mapped.
+effects and the pointer is globaly visible.
 
-
+The mappings are released with::
-::
 
 	void io_mapping_unmap(void *vaddr)
 
-This works like io_mapping_unmap_atomic, except it is used
+Use for pages mapped with io_mapping_map_wc().
-for pages mapped with io_mapping_map_wc.
 
 At driver close time, the io_mapping object must be freed::
 

include/linux/io-mapping.h

@@ -82,6 +82,21 @@ io_mapping_unmap_atomic(void __iomem *vaddr)
 	preempt_enable();
 }
 
+static inline void __iomem *
+io_mapping_map_local_wc(struct io_mapping *mapping, unsigned long offset)
+{
+	resource_size_t phys_addr;
+
+	BUG_ON(offset >= mapping->size);
+	phys_addr = mapping->base + offset;
+	return __iomap_local_pfn_prot(PHYS_PFN(phys_addr), mapping->prot);
+}
+
+static inline void io_mapping_unmap_local(void __iomem *vaddr)
+{
+	kunmap_local_indexed((void __force *)vaddr);
+}
+
 static inline void __iomem *
 io_mapping_map_wc(struct io_mapping *mapping,
 		  unsigned long offset,
@@ -101,7 +116,7 @@ io_mapping_unmap(void __iomem *vaddr)
 	iounmap(vaddr);
 }
 
-#else
+#else  /* HAVE_ATOMIC_IOMAP */
 
 #include <linux/uaccess.h>
 
@@ -166,7 +181,18 @@ io_mapping_unmap_atomic(void __iomem *vaddr)
 	preempt_enable();
 }
 
-#endif /* HAVE_ATOMIC_IOMAP */
+static inline void __iomem *
+io_mapping_map_local_wc(struct io_mapping *mapping, unsigned long offset)
+{
+	return io_mapping_map_wc(mapping, offset, PAGE_SIZE);
+}
+
+static inline void io_mapping_unmap_local(void __iomem *vaddr)
+{
+	io_mapping_unmap(vaddr);
+}
+
+#endif /* !HAVE_ATOMIC_IOMAP */
 
 static inline struct io_mapping *
 io_mapping_create_wc(resource_size_t base,