In my opinion, APFS does not seem to improve upon ZFS in several key areas (compression, sending/receiving snapshots, dedup, etc.). Apple is reimplementing many features already implemented in OpenZFS, btrfs (which itself reimplemented a lot of ZFS features), BSD HAMMER, etc.
Maybe extending one of these existing filesystems to add any functionality Apple needs on top of its existing features (and, hopefully, contributing that back to the open source implementation) would cost more person-hours than implementing APFS from scratch. Maybe not.
Either way, we will now have yet another filesystem to contend with, implement in non-Darwin kernels (maybe), and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices. Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+. It just seems wasteful of everyone's time to me.
>and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices.
What operating systems describe tremselves as being "compatible with Apple devices"
> Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+.
Who else actually "supports" HFS+ ? Sure there are linux "ports" based on the spec but nobody claims them as being "supported".
Apple would have had to continue supporting HFS+ whether they chose to implement ZFS, btrfs or HAMMER.
>It just seems wasteful of everyone's time to me.
I don't know how Apple writing their own filesystem is wasteful of anybody else's time ( except possibly Apple's and/or Disk utility software for vendors for OS X)
> What operating systems describe tremselves as being "compatible with Apple devices"
I was referring to the Linux kernel modules implementing HFS+ and other Apple FSes.
> Who else actually "supports" HFS+ ? Sure there are linux "ports" based on the spec but nobody claims them as being "supported".
Yes, by support I meant other developers who want to be able to read and write to devices in APFS format.
> Apple would have had to continue supporting HFS+ whether they chose to implement ZFS, btrfs or HAMMER.
Yes, Apple would have to continue supporting HFS+, but other kernel developers would not have to port yet another filesystem (APFS) with all of its own quirks; and, who knows, maybe it would be less work for Apple to inherit ZFS/btrfs/HAMMER/some other filesystem's solutions to some of the same problems they're trying to solve from scratch here. My point was more that by reinventing the wheel to implement some of these features, they've created not just more work for themselves potentially, but more for the open source kernel development community as well in the long run.
> I don't know how Apple writing their own filesystem is wasteful of anybody else's time ( except possibly Apple's and/or Disk utility software for vendors for OS X)
APFS will find its way to external HDDs/SSDs/flash drives, etc., then in order to read those filesystems someone else will have to port it to any other devices/readers of that device/FS.
> The standard is the interface ( POSIX / SUS ) and unless APFS breaks that how is this applicable ?
I didn't mention POSIX, VFS, or filesystem _interfaces_. The analogy to the XKCD strip was that we already had N filesystems that have a large subset of (or in some cases superset of) the features of APFS as of right now, now we have N+1 complex filesystems to contend with and port and interoperate with in other kernels/OSes (mainly Linux + non-Darwin BSDs).
This may just be the price of progress, which is fine. I think it'll be fantastic if Apple makes progress in this area and improves upon the work of others. The developer seemed to be ignoring history so as not to "taint" himself (did he mean IP/legally tainted?), which is slightly worrying to me.
I hope Apple open sources their implementation under a BSD/GPL dual license to make it easier for others to port it directly into other kernels, rather than having to reimplement it themselves.
Classic comic, but I don't think it applies. APFS looks intended to solve Apple's product problems really well, and it doesn't even try to be a filesystem for everyone.
Apple has said from time to time that they're all about owning and controlling the key technologies that go into their products. APFS makes a lot of sense from that perspective, and this seems one of those cases where going their own way is better than importing someone else's constraints. ZFS on an Apple Watch? LOL.
I would not be surprised if one could write a ZFS implementation optimized for more constrained devices. If you already know you are going to a have flash storage you can probably ditch some of the N layers of cache you see in common ZFS implementations. Not that ZFS is a one size fits all, but the file systems specification could be implemented in more than one way.
If you assume Apple cares about having a disk format in common with other platforms, sure, I'd agree that's probably possible. But I don't think they do; they seem to care a lot more about things like a unified codebase across their platforms, the energy-efficiency initiatives they've been pushing for a few years, owning the key tech in the products, etc.
One slide in the WWDC talk deck showed a bunch of divergent Apple storage technologies across all their platforms that are being replaced by APFS. If ZFS has to fork into weird variants to run well on the phone or watch, that seems less appealing than a single codebase optimized for just the stuff Apple products do.
I was reacting to the idea of APFS for macOS, as well as having yet another filesystem to deal with on external media that interacts with multiple computers (HDDs/SSDs/USB flash drives/etc.).
Is moving data between computers that way a thing that non-technical people do often? FAT-formatted USB sticks seem to be good enough for that, but e-mail/Dropbox/file sharing/cloud sharing/AirDrop have much better UX for the average person.
Yes, it is a thing people do. The problem is, those non-technical people do not understand on-disk formats. (Nowadays, most USB sticks come preformatted as ExFAT.) There are also offline and low bandwidth situations. In healthcare it's common too thanks to HIPAA and nervous hospitals: let's say a patient wants to transfer a set of MRI or CT scan images/videos (typically provided on CD or DVD, in which case it's ISO9660, but sometimes USB stick - hopefully ExFAT but sometimes worse).
APFS is targeted to run on low-powered devices like the Apple Watch. It may be that the alternatives cannot be made suitable for such devices: ZFS is famously memory hungry, HAMMER says it's not designed for < 50GB devices.
ZFS requires a lot of ram to enable on line deduplication.
From Freebsd Mastery: ZFS Pg 135
"For a rough-and-dirty approximation, you can assume that 1 TB of deduplicated data
uses about 5 GB of RAM. You can more closely approximate memory needs for your
particular data by looking at your data pool and doing some math. We recommend
always doing the math and computing how much RAM your data needs, then using the
most pessimistic result. If the math gives you a number above 5 GB, use your math. If
not, assume 5 GB per terabyte."
Otherwise I think this is like the myth that ZFS requires expensive ECC ram whereas ECC ram is recommended for any filesystem and zfs needs it no more nor less.
From the 2nd edition of "The Design and Implementation of the FreeBSD Operating System"
ZFS, Pg 549
"
However it is not designed for or well suited to run on resource constrained systems using 32 bit CPUs with less than
8 Gbyte of memory and one small nearly full disk, which is typical of many embedded systems
"
Even mobile cpus will be all 64 bit before long and it seemed to run great with 4gb others reported success on systems with 2gb. Meanwhile ram available in all sorts of devices is ever increasing.
What fs a laptop/workstation uses shouldn't be determined by whats suitable for a watch in 2016.
Is ZFS lightweight enough to run on the watch? Their ultimate goal was to have a file system efficient enough and flexible enough to run across all of their OS's - MacOS, iOS, tvOs, and watchOS
The Apple watch has 8 GB of storage and 512 Mb of ram; I don't think that's an unreasonable ratio. Most people discussing zfs memory use have large arrays or deduplication enabled; I'm not sure why dedup would be very useful on a watch. Maybe CPU for checksums is an issue? Apple could probably add acceleration for the checksum algorithm/use an algorithm that was faster/skip the checksums on read + do a sweep when plugged in and fully charged. An 8gb sad only takes minutes to do a complete read, as opposed to a large rotational drive, so you would likely be able to do a full scan in a reasonable amount of time.
Unclear, but a member of the ZFS development team at Apple told me that Giampaolo complained that ZFS would never work on the phone. So the team demonstrated it working on the phone. I know of no obstacles to making it work on the watch other than engineering.
So they still have a team working on ZFS? Could the anonymous "ilovezfs" involved with OpenZFSonOSX and coming from a California IP address be part of this team?
> Either way, we will now have yet another filesystem to contend with, implement in non-Darwin kernels (maybe), and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices.
On the other hand, if Apple decides to open source the APFS implementation (hard to tell what their plans are from current statements, but I'm holding out hope), it'll probably be under a permissive license that allows porting to Linux. The implementation is in C (not C++) so porting is probably generally feasible. Compare to ZFS, which, even if some distros have finally started shipping it, will never quite be free of licensing issues unless Oracle does a 180.
Maybe extending one of these existing filesystems to add any functionality Apple needs on top of its existing features (and, hopefully, contributing that back to the open source implementation) would cost more person-hours than implementing APFS from scratch. Maybe not.
Either way, we will now have yet another filesystem to contend with, implement in non-Darwin kernels (maybe), and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices. Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+. It just seems wasteful of everyone's time to me.
Also: https://xkcd.com/927/