Talk:Portage TMPDIR on tmpfs
Before creating a discussion or leaving a comment, please read about using talk pages. To create a new discussion, click here. Comments on an existing discussion should be signed using
~~~~:
A comment [[User:Larry|Larry]] 13:52, 13 May 2024 (UTC) : A reply [[User:Sally|Sally]] 03:38, 26 January 2025 (UTC) :: Your reply ~~~~
Redundant for systemd users?
tmpfs is already mounted on /tmp thanks to /usr/lib/systemd/system/tmp.mount on systems using Systemd. Therefore, don't Systemd users with default PORTAGE_TMPDIR automatically build in RAM? --Shoaloak (talk) 21:30, 31 May 2017 (UTC)
- Hi Shoaloak , this article is not mentioning /tmp, it is mentioning some directories under /var/tmp/ because this is where Portage stores its temporary directories (/var/tmp/portage for example). Hope this helps. --Maffblaster (talk) 22:30, 31 May 2017 (UTC)
Something about TMPDIR that I wanted to know when reading this page.
Note that the value of Portage's TMPDIR used here is the default value /var/tmp and can be verified by executing emerge --info|grep ^PORTAGE_TMPDIR . Portage does it's magic in a subdir called portage, hence why /var/tmp/portage is used above. --EmanueLczirai (talk) 17:55, 6 January 2015 (UTC)
- The article has reflected that for a while. Count it fixed. :) --Maffblaster (talk) 16:38, 8 June 2016 (UTC)
tmpfs within tmpfs
Worth noting that if /var/tmp/ is already tmpfs and you still want to mount /var/tmp/portage as tmpfs from /etc/fstab then the option x-mount.mkdir will help(without it /vat/tmp/portage/ folder won't exist and mount will fail). It may look something like this for example(in my case):
/etc/fstabtmpfs fstab exampletmpfs /var/tmp tmpfs rw,nosuid,noatime,nodev,size=7G,mode=1777 0 0
tmpfs /var/tmp/portage tmpfs rw,nosuid,noatime,nodev,size=7G,mode=775,uid=portage,gid=portage,x-mount.mkdir=775 0 0
--EmanueLczirai (talk) 05:48, 15 January 2015 (UTC)
- Excellent tip. I'll note it in the main article for now, but please feel free to add things like this yourself. The community would love your tips! :) --Maffblaster (talk) 05:32, 22 March 2017 (UTC)
Page move
Unless there are objections I believe it would be good to move this article to be a sub-article of Portage. This article is directly connected to Portage and (to me) it does not make too much sense to leave it dis-conjoined from the Portage article. The new location would either be Portage/TMPDIR or Portage/TMPDIR on tmpfs. We can either choose to call it the variable game, for simply use the suffix of the currently existing name and move it under Portage. Let me know if there are any objects and the reasoning behind them. I'll clean up any links that would link to the old location. --Maffblaster (talk) 15:58, 27 May 2015 (UTC)
- You seem to like attaching subpages. IMHO having a reasonable page title is much more important and putting links on relevant places does not leave any need for subpages. ... On the other hand, subpages tend to have a long name that is hard to remember, so it may be more user-friendly to use them as little as possible. You can also organize pages with the category feature, which is more suitable for creating a hierarchical network of information. --Charles17 (talk) 16:28, 27 May 2015 (UTC)
- I'd rather think of it as a tree, or URL, or folder structure. It's the same thing for the /etc/portage article and other articles. It seems to me that it's easier for users to find related content if it's organized with similar URLs, like a normal website (not a Wiki). The Wiki includes a nice search function, but that doesn't mean people should be forced to use it in order to find the pages they are looking for.
- Thinking of hierarchical structure, to which article Overlay or /etc/portage/repos.conf if not to both of them would you attach that other Managing_repositories_from_make.conf|new one as a subarticle? --Charles17 (talk) 17:23, 27 May 2015 (UTC)
- To support my point, there's the Xfce article and the guide version of the article is Xfce/Guide rather than Xcfe Guide (an entirely separate article). If the reader wants to get deeper information on an article they can simply type /Guide on the end of the software in question to get a longer, deeper article.
- I've been trying to keep a theme of connecting articles if they only related to a single piece of software. Since this article directly deals with TMPDIR exclusively for Portage, it makes a lot of sense to me to make the move. Categories can still be used if you'd like in order to link things. I'm not saying that we have to move it to be a sub-article of Portage, but I definitely think it needs some kind of title change to help readers more easily find it. --Maffblaster (talk) 16:46, 27 May 2015 (UTC)
- Charles old buddy, old pal. I'd still like to make this a sub-article of Portage, but I be happy with how it is currently and mark this discussion as closed. :) --Maffblaster (talk) 16:42, 8 June 2016 (UTC)
Not enough 2GB on tmpfs info
For virtualbox-5.0.20 not enough 2GB on tmpfs (/var/tmp/portage) for me.
app-emulation/wine dev-lang/mono dev-lang/rust
--Cronolio (talk) 03:30, 30 June 2016 (UTC)
- Sure. Lets increase the size to 4GBs. That should be enough. It is not enough then increase it more. :) --Maffblaster (talk) 05:30, 22 March 2017 (UTC)
Running out of inodes on tmpfs
Is what actually happened in my case, when trying to build www-client/chromium-57.0.2987.98 (and 56 before that). Having 16G of RAM, I've had allocated 4G for PORTAGE_TMPDIR on tmpfs. But the emerge complained on the lack of free space during unpack. Increasing tmpfs size gradually - 5G, 6G, 8 .. 12G did nothing to help. Surprisingly(for me), actual size of unpacked data never exceeded ~2G - the point after which out-of-space message started showing up. That triggered some suspicion. So, having found some docs on the topic... tmpfs.txt I've applied the mount' option nr_inodes=0 to fix possible lack of inodes. After that, unpack went smoothly and build completed successfully. There's possible downside to having nr_indodes with value equal "0"(unlimited amount of possible inodes created), so actual value may be amended to your taste. Some more information on that here http://serverfault.com/questions/524419/why-dont-linux-distributions-default-to-mounting-tmpfs-with-infinite-inodes
Asking mods to reflect my point in troubleshooting section(sorry, my wiki-fu is not nearly strong enough to handle it neatly). — The preceding unsigned comment was added by D-ion (talk • contribs) March 23, 2017
- Hi, D-ion , this is very interesting. Thanks for the tip! Can you integrate this into a Troubleshooting section of the article? Also, be sure to sign your comments to discussion pages (see the little button in the edit toolbox). I will fix the first one for you. Kind regards, --Maffblaster (talk) 18:09, 23 March 2017 (UTC)
- D-ion , no problem. I see you added it to the Troubleshooting section. Thanks for contributing! I'll mark this discussion as closed. Kind regards, --Maffblaster (talk) 23:54, 27 March 2017 (UTC)
Memory usage for gcc-9 is > 4GB...
I just compiled stable gcc on ppc64, and my tmpfs is set to 4G, and the build failed with "No space left on device" while being 3983m big.
sys-devel/gcc-9.3.0-r1:9.3.0::gentoo [8.3.0-r1:8.3.0::gentoo] USE="altivec (cxx) fortran nls nptl openmp pch (pie) sanitize ssp (-ada) -d% -debug -doc (-fixed-point) -go -graphite (-hardened) -jit (-libssp) -lto% (-multilib) -objc -objc++ -objc-gc -pgo -systemtap -test -vanilla (-vtv) (-mpx%)"
Since neither Java nor Objective-C are enabled, I changed the article accordingly. I don't think it's specific to ppc64... Luttztfz (talk) 04:39, 17 September 2020 (UTC)
- Thank you! --Maffblaster (talk) 17:15, 17 September 2020 (UTC)
Advantages and disadvantages of TMPDIR on tmpfs?
I just noticed the important note on top of the article, that there is no speed gain with "-pipe" in the CFLAGS. Okay, so why should I then bother to use tmpfs at all?
Shouldn't there be at least a hint towards advantages of TMPDIR on tmpfs for portage? Especially for SSDs, isn't minimizing writes a good idea, hence tmpfs would provide at least the possibility of avoiding unnecessary writes on the mounted media? I know that tmpfs is page-able, meaning if memory gets full it will be swapped out, thus writing to whatever device is used as swap. ramfs would be non-page-able, but it isn't used much anymore. (Is it even still in the kernel?)
The point being: a note towards advantages/benefits would be nice, not only in the article SSD#Reducing amount of writes...
Luttztfz (talk) 09:27, 12 March 2024 (UTC)
- According to triffid_hunter on reddit, -pipe only affects c -> preprocessor -> compiler -> assembler, but not linking or any other things required to assemble a mergeable package image, which would be read from disk/tmpfs SigHunter (talk) 12:44, 12 March 2024 (UTC)
- File items are highly likely to still be present in the file cache however, and wear on modern SSDs isn't really a thing any longer. Users with lots of RAM to spare and/or still making use of spinning disks would, in my opinion, get more benefit from a zram device given how well source code compresses and how fast lz4/zstd are. This would give fewer problems with the larger source code bases, thus less need for a separate env configuration for falling back to disk. Ninpo (talk) 13:00, 12 March 2024 (UTC)
- I tried to set up zram to replace tmpfs, and it actually works well. Thanks for this hint. On a Ryzen 5800H notebook (8 cores) there is a notable delay after uncompressing the source tree to the zram device, which is because of the use of zstd compression I configured. lzo or lz4, which I didn't test, should be faster.
- I added my zram configuration as examples to the Zram article under #Using_systemd_zram-generator.
- Should this be incorporated into this article as well, or into the SSD article? Is there a well tested configuration for this? What are the caveats? Luttztfz (talk) 15:24, 15 March 2024 (UTC)
- Update: I added a note for "Portage TMPDIR on zram"... Luttztfz (talk) 20:31, 19 March 2024 (UTC)
Unhealthy levels of fearfulness in the intro paragraph
Who wrote this? Somebody who just came over from Ubuntu?
I tested using a tmpfs extensively. And even on a 1GB RAM (!!) single-board computer with a (SATA) SSD, it provided a massive speedup. (gcc lines flying by instead of slow-dripping in.) Aside from the reduced wear, of course. (Portage is a massive flash memory churner, even with -pipe.)
The trick is to of course take out large packages (via package.env) and point them to the non-RAM TMPDIR. (Most packages only take a couple of MB.) And set a size limit on the tmpfs. Or just buy more RAM. Come on, 16GB are 25€ right now! There is no reason to have that little RAM. We’re not Apple laptops! ;)
Besides: THIS! IS! SPARTA! GENTOO! … The whole point is that it is for pros, who want control and power, and can and want to handle it. In fact it is the last bastion us computer literates have! So arguing about “complexity” like we are children who need rubber padding and plastic knives and having their food chewed for them, their will wanted for them, and saving by the big condescending coprorat [sic] nannies… If somebody is afraid of setting this up, or of patching their kernel or scripting the CLI, then why the hell are they using Gentoo, and aren’t used via an iPad? Maybe ask their legal guardian (Apple, probably) to pull them away from the ka-chunk ka-chunk industrial machinery, would ya?
— Evi1m4chine (talk) 00:34, 16 November 2024 (UTC)
- It's hardly unnecessary. Your boast about using tmpfs for TMPDIR on a 1GB system already has the caveat that you need to exclude a whole bunch of packages from being extracted into the TMPFS, packages that would arguably benefit most from source being in memory. This is the complexity of configuration being introduced that the article highlights. You say "buy more RAM", this doesn't exclude the fact we have many users that can't, or won't, making the advice in the article still very relevant.
- Starting anything with the hyperbole your first comment did does not lead us down a productive path for discussion. Please focus on substantive arguments instead. We're all using Gentoo, we all want the docs to be good, and so on. Stick to the substantive points of disagreement. Thanks. I'll also note that many of the arguments you use here could be applied to removing any nuance, warnings, or caveats from most articles. That obviously is not helpful. --Sam (talk) 01:16, 16 November 2024 (UTC)
- To echo Sam's words lets start again.
- Can you provide some benchmarks for systems where you see a cost to rewards benefit and also some recent reports of wear issue on modern solid state drives.
- Once we have this then we can discuss if the warnings are too strongly worded as is.
- Immolo (talk) 05:19, 16 November 2024 (UTC)
- Sam seems to have expressed very well how we strive to work productively here. Just wanted add a to link to the code of conduct, because that framework is a good base for working effectively together on the wiki. Thanks. -- Ris (talk) 08:25, 16 November 2024 (UTC)
- @Evi1m4chine: you know damn well who wrote this, because there is a version history. And if we're all pros here, then you are too, and you can look in the wiki version history for yourself. Yes?
- So, it seems that I wrote this... partly. It all started when I found the note that
-pipewould suffice, and altered the intro as indicated by this note, this edit. Then Kangie changed it again, saying that it's all very hard to quantify, this (next) edit. So I'm thinking "absolutely correct", but this article is about Portage on TMPFS, and all we do is to discourage the reader and to say, that he shouldn't do it, which is very negative. Isn't there a–even if theoretical–basis for all of this "in RAM" portage build process? Yes, but theory isn't reality, which is what I wanted to convey with those (next 9) edits... - You yourself also ask if there are benchmarks. You have any? If not, well... isn't it all very hard to quantify?
- I myself ran into the situation of going into massive swapping because I had set-up portage on TMPFS "too extensively". RAM is needed also for the programs you use and for the compiler and linker, and if you use more threads even more so. In the end, the well-intentioned benefit turned into a massive penalty, because (as with probably many users) my swap-space was also on the SSD.
- As you can see in the discussion above, #Advantages and disadvantages of TMPDIR on tmpfs?, I just recently switched to "Portage on ZRAM", which was a massive gain memory-wise (it uses way less memory with Portage, due to source files being well compressible), but only because my system is fast. On an old system, where I didn't try ZRAM (yet; but I know how slow compression is, e.g. using xz, compared to my current system), this is probably the opposite of a speed-gain.
- And why would you think that you know what circumstances are present for every possible system of "us Gentoo pros"? I use(d) Gentoo on a variety of different systems, from x86-32, ppc32, ppc64, arm and x86-64, and for some systems RAM simply wasn't upgradeable (either not at all, or not beyond a certain maximum, like a Power Mac G4 is often maxed out at 2 or 4 GB), and compression performance isn't good either. Depending on what you intend to do, TMPFS may not be a great option on such a system if you accidentally run into this swapping condition I experienced and already described above. But, if you only compile small stuff, it may be the right thing to do–for you! But maybe not for others, which was the reason for me to include both (theoretical AND practical) advantages as well as disadvantages (because: hard to quantify!).
- So, what does this mean for this article?
- At its current state (as of this writing: this state), it again starts off with a long list of reasons not to use Portage on TMPFS. I find this very counter-intuitive. I would want this in a warning, but not as the main text in the intro, at least not with some benefits as well, even if they are theoretical and the practical benefit may vary for every user and use-case.
- And, important: I'm not saying that "my intro" was better or anything like that.
- Just my 2¢. Trying to be productive as a decades old Gentoo user. Luttztfz (talk) 16:47, 16 November 2024 (UTC)