File systems aren’t discussed on XDA or any other forum all too often. The topic involves a lot of low-level development, so developers prefer to focus on applications, ROMs, or kernels. Nevertheless, the file system is a vital part of every storage. It’s the technology that outlines how data in your device’s storage is stored and retrieved. There are many different kinds of file systems—each with their own pros and cons—and picking one over another can make a huge difference in stability and performance. So how do OEMs make that choice? I had the pleasure to talk with Stan Dimitriev, the PR & Communications Manager at Tuxera, a Finnish company that provides file system solutions to many big OEMs.
Q: Can you introduce yourself and Tuxera?
A: I’m Stan Dmitriev. Ever since I got my first Android phone (HTC EVO 3D), I’ve become an active fan of the XDA community. Let’s just say I’ve installed a custom ROM and voided my warranty on day one (Stan.Dmitriev is a Forum Member on XDA). A couple of years ago, I co-founded a startup project called Corgi for Feedly, at that time the XDA community helped us a lot with defining and developing the application.
Currently I’m the PR & Communications Manager at Tuxera, which is the leading embedded storage software, file systems to be precise, company on the market. Our software powers more than one billion devices and can be found in the latest flagship phones, cars, routers, drones, and cameras. The story of the company started with NTFS-3G, when our President and CTO Szabolcs “Szaka” Szakactics made NTFS to work with Linux devices. Currently, we develop our own file systems and implementations both for external and embedded storage and work with most of the automotive companies out there as well as many smartphone companies.
Here at Tuxera we quite often visit the XDA forums, especially when troubleshooting a bricked device or finding a custom ROM / kernel to play with. Many of my colleagues frequently visit XDA to read about the latest deep tech news around Android and Linux ecosystems.
Q: File Systems are an integral part of our devices, but very few people know about their role, improvements, strengths, and weaknesses. What’s the gift of it, and why should more people take interest in the file systems holding together their data?
A: File Systems are a vital element of the Linux kernel that is responsible for all of the interactions between the device and its storage. Every time you take a picture, open an app, or watch a video – the file system plays a core role for the OS to access and store all of the files. File systems weren’t so exciting for some time, the main reason is that the bottleneck in performance was coming from the storage.
But as flash speeds are getting extremely fast, especially with introduction of UFS, it is now the software that needs to be modernized to keep up with the flash performance. Not only can the file system affect the storage speeds, but it also plays an important role in maintaining the phone performance in the long run. If done wrong, the storage will become fragmented, a phenomenon in which an inefficient use of storage can reduce the performance of the device. Reducing such possibility to a minimum, is an important characteristic of a file system.
Q: What is F2FS (primer), and why did it start making its way to Android? Which OEMs were leading the charge?
A: F2FS is an open source file system, with the main pitch that it is designed for flash storage architecture. NAND storage (flash) is used in all of the current Android phones usually as eMMC or UFS solutions. The project was developed by Samsung initially and has gained strong interest from some of the mobile OEMs. We don’t have the data on which OEMs started this shift. But from what I know, personally, Motorola was the first, with many other OEMs like OnePlus and Huawei following this shift shortly.
Q: What were the advantages it provided? OEMs cite faster storage operations and UI performance, but are there any other advantages?
A: The pitch was quite simple, F2FS brings higher performance to flash based storage. Your system would boot faster (faster read speeds), the write speeds would be higher, allowing to record heavy 4K, higher frame rate slow-motion, and 360-degree videos. Additionally, the latency between the system and the storage would be ultra-low, meaning that the animation and opening apps would feel much snappier.
Q: The issues, specifically, seem to center around degrading performance. Could you explain that in greater depth? What’s the cause?
A: The idea of F2FS was indeed great, to achieve the fastest possible performance for flash devices. But it is also important to focus on the performance in the long-run. Hardware can always be fast out of the box, but maintaining these speeds continuously is much tougher challenge. For an Android use case the storage of the device is almost full most of the time, and there are thousands of read write operations happening sometimes even in a second. Many of the popular apps run services in the background, this means that the app is doing some read/write operations even when you don’t open it. If done wrong, write operations can drastically increase the fragmentation of storage, making every next read/write operation even slower.
We see storage fragmentation as one of the causes for degrading performance in Android devices. Now what has to be determined is, whether the issues are caused by hardware compatibility and how the file system performs with storage from various OEMs, or whether the F2FS architecture focuses on initial speeds, and this affects the long-term performance. The issues can be also caused by bugs, as the file system is still in development.
Q: Are there any other performance or reliability issues that are less well-known?
A: One interesting thing we found is that F2FS has an unexpected overhead, at least couple of hundreds of MB of your storage for caching and storage maintenance. This definitely makes sense if you have 64GB of storage or more, but on a budget phones with 8GB, 16GB ROM sizes, this might be a problem.
The main finding is that the current implementation of F2FS is great for read-only environments when you want to achieve higher speeds. F2FS needs to focus on improving its long-term storage performance in a high interaction with small files, as this is what happens in the Android environment.
This is in a sense a battle of theories. You can either envision that the hardware performance is so fast, that the degradation won’t be an issue. Or you look at it the opposite way, where the hardware is so fast, that having a bit slower performance that can be maintained in the long run, is what should be achieved. F2FS chose the first approach, and there are no wrong options here, it is an important experiment to make, and a learning curve for the whole industry. That being said, what we are currently seeing, is that long-term performance should be the main focus right now.
Q: Some OEMs have walked back their use of F2FS, while others like Huawei still support F2FS on recent devices. Do you think there’s a trend either way? (Slow adoption, or complete abandonment)
A: There are multiple scenarios why OEMs would continue using F2FS.
An OEM could have their own implementation of F2FS, that brings some fixes to the known issues.
An OEM could have very good ties with the flash storage manufacturer, since having access to the FTL layer of NAND and tuning the file system to work well with the hardware can bring significant improvements, including to the fragmentation of flash storage.
An OEM could prefer to focus on the initial performance of the phone, as the average lifecycle of a smartphone is around 2 years, and as the phone gets slow they will already be buying a new one.
An OEM could also have some defragmentation service running, this can partially solve the issues of F2FS.
These are some of the possible scenarios, at the same time, we are seeing that some OEMs are already moving back to using Ext4, for example OnePlus 3T was using F2FS, but all the newer OnePlus models use Ext4.
Q: F2FS is currently in a need of some on-the-go defragmentation solution, that could solve the issue that is currently appearing with F2FS.
A: Defragmentation tools have existed for some time, the main advantage is that they can reallocate/rewrite the memory blocks to reduce the storage fragmentation. But the issue with this approach, is that you rewrite the flash memory cells, which reduces the lifetime of the storage. The best scenario would be to write data intelligently in the first place. Doing damage control is not the go-to solution.
Think of it like making a mess in a room by throwing a lot of stuff all around, and then starting to arrange them in to the drawers. Why not do that straight away?
Q: Have OEMs managed to counter-act or address some of the issues holding back F2FS? For example, Huawei dismisses degrading performance and advertises a faster UX over time, in part thanks to their “AI”.
A: Well if an AI can analyze the storage behavior and rewrite the file system to work for a specific flash device, this can probably solve the issues of fragmentation. The AI can also analyze which apps or features the person uses mostly, and pre-load / store them accordingly – making the phone features feel snappier, that wouldn’t solve the issue of fragmentation though. My main assumption is that most of the optimizations are user-space related, and not some deep tech file system optimizations.
Q: What are some other exciting file system developments that we should be aware of? How would they impact the user experience, even if on a subtle or low level?
A: F2FS and Ext4 are still very vital and are the go-to file systems for open source. Ext4 is more mature and reliable but might limit the flash performance; F2FS is snappier but also more experimental. The whole situation where software becomes the bottleneck is what makes file systems exciting again. We are moving from the every second, to every millisecond counts environments and file systems will play an important role in this performance boost. The most exciting development for File Systems, would be to create a truly flash optimized solution that can maintain the performance of flash.
At Tuxera, we are developing Tuxera Flash File System, where we do things like hardware-specific software optimizations ensuring that the file system is optimized for a specific device, use case, and flash memory. Having a presence in automotive, we mainly focus on the long-term performance and degradation of the flash memory. This is more of the hardware x software type of optimization, that Steve Jobs used to talk about. Applying Machine Learning to create such “drivers” is something that we think could be really exciting for the whole storage industry.
Q: Our physical flash storage keeps getting faster as well, how do you see this fact impacting both the end-user experience and the future of file systems?
A: As hardware gets faster, software needs to ensure that it can keep up to these advanced hardware architectures, this is the main challenge for file systems to solve currently. You can have incredible read and write speed characteristics of the device, but if the software stack brings added latency – the user experience won’t feel that snappy. This is an exciting time for file system development, since there is a clear need for a faster, snappier, and yet more reliable technology for mobile.
But as flash gets faster, the main question becomes, what is the speed at which performance becomes irrelevant? Think of it like the Retina screen resolution, where a higher Pixel Density doesn’t really make the image crispier for the human eye.
Q: Thank you for your time.
A: Thanks!
from xda-developers https://ift.tt/2EoyyUP
via IFTTT
