cosmic_cheese
I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
That would make it unnecessary to pull in foreign dependencies in the overwhelming majority of projects. What few do get pulled in becomes lightweight, easily verifiable syntactic sugar or libraries with purposes too niche to be worth targeting.
Of course this approach can go wrong too. You could easily end up with a monster like Boost, but that comes down to project administration keeping creep under control and proper modular design.
yoyohello13reply
This was actually the main thing that put me off of Rust. I get the argument for a small std lib. I just also don’t agree it’s worth it. Go seems to handle having a batteries included standard lib just fine.
0cf8612b2e1ereply
On the other hand, there are some bad Go standard libraries that are frozen in time.
rirzereply
which, as painful as it may be, is ok. Better to have a safe functional stdlib library than a exposed external crate (which then asks the question; what's the replacement...)
bigstrat2003reply
Which isn't actually a problem. You can ignore the bad standard library and use something different.
pjmlpreply
Yeah, but I will take a not great library that works everywhere the compiler does, than be at the whims of which platforms are supported by 3rd party libraries.
I can use most of the clunky Python, Java, .NET and if it must be, Go, standard libraries, than hunting down for dependencies with platform tier support and such.
afdbcreidreply
People confuse what they want.
They do not want a big stdlib. The downsides are real (the stdlib cannot make breaking changes), and there are no upsides (except, maybe, for faster compilation, since std comes precompiled).
They want more official crates (e.g. `regex` and `libc` are official crates, maintained by the Rust project). And the Rust project does not oppose to that, it just doesn't have the funding.
Pannoniaereply
Who said a standard library can't make breaking changes? That used to be a norm some time ago.
There are upsides, your program is smaller, better security because a random kid can't pwn your deps, quality and interoperability. What's not to love?
josephgreply
> there are no upsides (except, maybe, for faster compilation
Another big reason to put something in std is providing types for cross-crate compatibility. If I want to pass a String from one crate to another, I’m glad that string is defined in std so there’s an obvious type we can both use in our APIs.
C - for example - does not have this luxury. Everyone makes their own string type, and C APIs all need to translate strings at the api boundary. It’s super annoying.
It would be nice if we had a standard way in rust to declare a type as serialisable. Right now lots of crates have a serde feature flag to do this with serde. But (a) they need to put this behind a feature flag, since it adds a dependency on serde. And (b) this doesn’t work with other serialisation libraries.
The other big one is futures. There’s still no futures executor in std. Async crates have to decide if they want to tie themselves to a single executor (like Tokio) or be compatible. There is no async stream api in std. No async file api. And so on. It’s a compatibility nightmare.
gigariteereply
This is not true. I want a big standard library and I am more than willing to deal with the downsides just like i have with python.
Batteries included ia the only sane way to do programming languages.
leecommamichaelreply
The Odin programming language does this! It has also decided not to provide a package manager.
Ygg2reply
Spoiler alert! JavaScript has no language provided package manager.
If Odin gets moderately successful someone will probably reinvent it.
leecommamichaelreply
NPM came along in 2010, Javascript was huge before that.
It was only when people wanted a common approach to shipping both in browser and "native" that this glitch happened. I believe a standard library and "batteries" would have abated it entirely or resulted in a slightly less-bad situation. I do think Cargo is a slightly less-bad situation in many ways, and that it can be done better.
hnlmorgreply
Go took the same approach and ended up having to implement a halfarsed one when everyone started implementing their own.
leecommamichaelreply
Has Go not had the most secure ecosystem?
What is your critique of their approach? Is it not the case that `go get` is the only one which doesn't even provide a way for the person downloading to run the downloaded code until it is actually executed by the consuming codebase? That seems pretty sound by comparison.
I guess you're saying, "they had to" meaning they should've seen the need and provided it? I'll say this to that (imagined) take; plenty of useful software was made without it, and they got the job done when they knew the absolute most about what a good solution would need. I totally understand being annoyed at the fact that this is the story of every evolution in Go, but I genuinely think they're picking good implementations when they decide on them.
mightyhamreply
Rust has a sizable and well featured standard library at this point. I think it would be absurd to claim that the base language is "near-unusable" if you are using it for system programming, which is its intended use case.
Huge standard libraries make sense for Java, Go, Apple platform etc. because they are developed by giant enterprises that can manage the overhead. Thinner modular systems are a more natural fit for open source development.
cosmic_cheesereply
It’s being used for a wide variety of other purposes however, and so perhaps it’s time to adapt to that reality.
While overhead is a real concern, I think it’s often blown out of proportion. Once a language achieves a certain baseline of stability and isn’t in constant flux and the standard library matures, changes become infrequent and maintenance load is low. The work is heavily front-loaded.
VorpalWayreply
Not at all true, look at the large parts of Python's standard library that are effectively unusable or irrelevant. And they even allow some breaking changes over time (unlike Rust). This is especially true when it comes to anything related to internet protocols or file formats, but there are other modules that have far better replacements on PyPi too (re vs regex for example).
kelnosreply
> It’s being used for a wide variety of other purposes however, and so perhaps it’s time to adapt to that reality.
Who is going to pay for the resources to make that happen, though? The reality is that the bulk of Rust contributions come from a fairly small group of people, many of whom are volunteers. Some are Mozilla employees, and they don't have all the funding in the world.
Yes, it would be great if every open source project could magically be able to meet the needs of all its users, but that just isn't how it works.
josephgreply
True. And the rust standard library is pretty good. But I really wish we had a good, fast, small futures executor in std. And accompanying async variants of File and Socket and so on.
Async rust is a jungle of weird compatibility questions. People treat async runtimes like sports teams. (I know I do). I wish it were more like Nodejs where async is just built in.
socalgal2reply
If rust has a well featured standard library, why do the rust docs need to download 741 crates?
tancop
We need effect based languages now. It's the only way to guarantee policies like no network, no file access, no unsafe code or FFI for a library before it even compiles. If anyone from epic is reading this please give us a timeline for open sorucing the Verse compiler.
In the mean time I think it's possible to hack Cargo and run all build scripts in a microVM. The blast radius will be limited to malicious code in the binary instead of uploading all your CI secrets and deleting the whole hard drive.
burnt-resistorreply
More than that, we need capability-based languages. No capability passed to it, no permission.
kibwenreply
First we need capability-based OSes, like we should have had decades ago if worse-is-better hadn't stuck us with Unix. I don't need to care whether or not a program was written in a capability-aware language if the OS fundamentally takes care of that for me.
ptxreply
FreeBSD does this with Capsicum: https://wiki.freebsd.org/Capsicum
grommzreply
HarmonyOS is a capability based OS. I don't know why anyone in China is still using Rust. The biggest social engineering hack was for the Rust team to convince developers that it is a safe language.
burnt-resistorreply
1. seL4 exists. I haven't seen much more formal and rigorous than it. Efforts have ported the Linux kernel and RTOS-things to it.
2. Add syscall(s) to POSIX, which encourages *nix adoption, with an API like the following:
- Drop capabilities for thread/process except allow list.
- Test if current thread/process has all listed capabilities.
- List should be able to represent capabilities and their attributes, much like a single unveil() OpenBSD call, maybe as an array of/pointer to struct.
- (Program crashes if syscall capability is missing.)
mike_hearnreply
Why not object capability languages: https://blog.plan99.net/why-not-capability-languages-a8e6cbd...
burnt-resistorreply
No. Object-oriented brings things many languages cannot use. Also, that was a tl;dr fluff blog with barely any examples.
quotemstrreply
MicroVM this, effects that --- can we discuss security without needing to invoke bay area buzzwords?
The idea is "least privilege", and we desperately need it in computing. The precise technical mechanism we use to achieve it is less important than committing to the idea that a dependency doesn't run with full privilege of its host program and an install script doesn't run with full privilege of the programmer.
The original Linux seccomp is old enough to drink. It's always been possible to do things like expand macros in a no-IO environment. Nobody's bothered to do it over the past two decades. Why would anyone bother in the next two decades?
insanitybitreply
The technical mechanism is exactly the issue to figure out, there's a reason why projects don't have this and it's because different implementations have different tradeoffs.
quotemstrreply
No, the reason people haven't been doing this -- and we've had technologies for ages -- is that it's a huge pain in the ass, a "tax", that it's hard to get developers inside a company to pay, much less participants in open source ecosystems.
Look at how snap, flatpak, etc. provoke people to just turn off security rather than deal with breakages.
A new language won't help because the problem is social, not technical.
FFS, you can't even get people to use filesystems via intermediate objects in today's languages. People think your language is broken if you don't have an ambient open(). You don't need a new language to enforce capability discipline. You need to whack people repeatedly with a cluebat until their laziness and brain damage abate.
insanitybitreply
https://github.com/insanitybit/witchy
This is why I'm building this language. It's capabilities based. Build scripts can't just do whatever the hell they want to, everything is auditable, and it layers its sandboxing.
It's for fun and anyone looking should understand that this is AI driven building with human driven design, but the goal is to demonstrate.
fnoef
Oh, so it's not only "JavaScript bad and npm bad". Apparently, if your language uses third party dependency registry, you are prone to malicious code, regardless if it's Javascript or not.
Use containers for development. And reduce the amount of third party deps you import into your projects. This is only going to get worse.
insanitybitreply
If it weren't a registry it would be ./configure scripts and makefiles. The issue is that sandboxing technology is kinda shit (especially x-plat) and languages don't build it in by default.
patmorgan23reply
Pulling in lots of dependencies creates this kind of risk regardless of the ecosystem. That being said in the JS/NPM world you tend to have a LOT more dependencies (especially indirect ones) than other languages. I saw someone do a cursory analysis and JS/Node projects tend to have 5x the number rust or ruby projects.
This is really a cultural problem not a technical one.
frollogastonreply
It's funny how left-pad was always brought up as a JS supply chain vuln when that wasn't even malware
jakubadamw
Cargo desperately needs sandboxing for build.rs scripts. It’s been attempted before, but didn’t go very far¹.
¹ https://rust-lang.github.io/goals/2024h2/sandboxed-build-scr...
Panzerschrekreply
Sandboxing for build scripts can't work properly. If you sandbox too much, some necessary stuff can't be done. If you sandbox too little, it has no practical value.
loboftareply
So let each build script define its own level of sandboxing and then users can determine whether they are okay with that level or not, e.g. `cargo build --sandbox-level=...`
jurgenburgenreply
That’s not solving the problem, that’s avoiding it by making it the users fault if they make a mistake.
amlutoreply
As an easy start, how about letting build scripts read /usr, read and write a temporary build directory, have some /tmp scratch space, and be allowed to write its final output artifact. No network and otherwise isolated from the rest of the system.
I would argue that, if a build script doesn’t work in the setting, then it doesn’t deserve to be installable by a default cargo command.
kibwenreply
Cargo is a cross-platform tool, so when it ships a sandboxing solution it will need to be a cross-platform solution, and because this is a security feature it needs to be bulletproof, so no half-measures like Docker. Something like a WASM runtime might fit the bill, though that will be much easier to get working for typical proc macros than for typical build scripts. If you only care about Unix, then you can do this yourself today by building code in your sandbox of choice.
krautsauerreply
https://news.ycombinator.com/item?id=49374811
(Oh and btw, proc macros also run arbitrary code.)
quotemstrreply
There's no good reason a proc macro can't run in a no-IO sandbox by default. None. Doesn't require a language change. Doesn't require some microvmcapabilityeffect BS. It requires looking people straight in the eye and saying "no" when they complain about needing to prompt for privileges.
weinzierlreply
Sandboxing just build.rs would only be
be a minor inconvenience for the attacker, nothing more. The attacker can always as easily compromise the binary you build and as soon as you run it (e.g. in a test) you are owned.
It would be a big pain for many that are in the unfortunate position to really need build scripts, though.
insanitybitreply
It would be more than a minor inconvenience. I can handle sandboxing my tests and production infra, but I can't handle sandboxing build scripts because I don't own that code in any sense.
Aurornisreply
I imagine it would be sandboxed by default with an escape hatch to run build scripts outside of the sandbox with user verification. It makes people stop and think about what’s happening. Not perfect, but it does help. When working on JS ecosystem projects I manually approve build scripts and spend some time researching dependencies with build scripts to see if I can avoid running the build script. Some people will ignore it and run everything, but it’s a huge step in the right direction to make it operator-decided.
kibwenreply
At the very least, it wouldn't be overly onerous when adding a dependency that requires a build script to require an opt-in via Cargo.toml, e.g. `build-script = true`. You'd make it viral so that any transitive dependency that requires a build script would affect its parent, then add the key as defaulting to `true` so as to not break backwards-compatibility, then switch the default to be more restrictive over a new edition. (This same key could be used to prevent proc-macros from having arbitrary system access as well, where by default proc macros could be compiled to WASM and run in a WASM sandbox and treated as pure functions.)
