Rust: What the fuck is a borrow checker and why is it yelling at me?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

I could never get valgrind to work but I think I got Heaptrack to work. It looks like I did actually fix a leak from it as something it showed went away after an alteration that I realized was necessary (not reclaiming a bunch of char*'s in vectors I created for a C call). However, there's still some real moonbeams flying out of this report. Take this:

code:

        let wchar_id = widestring::WideCString::from_str(unsafe_cstring
            .into_string()
            .unwrap())     <-- What I have created here apparently leaks
            .unwrap();

What's up with that?

It also looks like some vectors I'm making are dangling and causing leaks. Paraphrased code:

code:

        let (mut system_keys, mut system_values) = get_system_common();

        // Add more stuff to system_keys, system_values...

        let mut raw_keys: Vec<*const u32> = Vec::new();
        let mut raw_vals: Vec<*const u32> = Vec::new();

        for key in system_keys {
            raw_keys.push(key.into_raw())
        }

        for val in system_values {
            raw_vals.push(val.into_raw())
        }

It looks like when I iterate system_keys and values, I implicitly call into_iter() which moves ownership. If I switch to `for key in &system_keys`, my casting of key screws up:

code:

479 |             raw_keys.push(key.into_raw());
    |                           ^^^^^^^^^^^^^^ move occurs because `*key` has type `UCString<u32>`, which does not implement the `Copy` trait

What's going on there and how can I keep this stuff properly clean?

Edit: I'm also just generally surprised I'm leaking like this. Also, I should note later in the code, all those into_raw() calls do get reclaimed and dropped. I'm not leaking that stuff.

Rocko Bonaparte fucked around with this message at 08:01 on Jan 20, 2023

# ? Jan 20, 2023 06:55

Adbot: ADBOT LOVES YOU

# ? Apr 27, 2024 01:12

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Unless you're specifically trying to pass ownership of those allocations elsewhere, do not call into_raw. Instead, call as_ptr and make sure you don't invalidate those pointers until whatever downstream code is done reading through them.

# ? Jan 20, 2023 07:49

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Ahhhhh you wrote back before I could qualify that. Those into_raw calls are getting properly reclaimed. I'm not leaking those.

(I was leaking those and already fixed it beforehand).

# ? Jan 20, 2023 08:02

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Overnight update after getting some sleep, I ended up just using as_ptr anyways. The fundamental problem with the casting was that into_raw() was giving me a mutable but I needed a constant reference for the C API signature, and I couldn't figure out how to do that.

I still don't really know why I'm leaking the way I am in all of these calls. I am just making a theory that I am loaning mutable stuff and since I'm not overtly reclaiming it, Rust is deciding not to drop it at the end of the function/scope. What I'm trying to do is add drops to stuff that is strangely leaking and seeing if the compiler writes me hate mail, fixing that, and then getting rid of the drop calls.

# ? Jan 20, 2023 17:45

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

An explicit drop call is not likely to fix a leak, since everything in a scope gets dropped when the scope ends anyway. You're probably releasing ownership somewhere and not reclaiming it with from_raw or equivalent, assuming you're not outright calling mem::forget.

# ? Jan 21, 2023 00:56

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Ralith posted:

An explicit drop call is not likely to fix a leak, since everything in a scope gets dropped when the scope ends anyway.

Yes I wasn't thinking so. The idea was that if I put it there, and I get yelled at because the thing I'm trying to drop was borrowed, then I was taking it that whatever borrowed it wasn't managing it. I had on vector getting iterated that was borrowing it, and a leak went away when I started to use a reference of it. I saw it when I tried to drop the vector even though I had no normal reason to do it otherwise.

On the other hand, it was done with a few other things in a commit so I might be completely wrong. I'd be curious if everybody says that's bullshit because I thought it was odd and I don't want to learn a wrong thing.

Edit: I'll see if I can get a bit of time next week to try some isolated code out with vectors and some of this handling to see if I can replicate that leak in something completely sharable and self-contained.

Rocko Bonaparte fucked around with this message at 06:20 on Jan 21, 2023

# ? Jan 21, 2023 02:10

teen phone cutie: Jun 18, 2012; last year i rewrote something awful from scratch because i hate myself

my work gives learning stipends and i want to learn rust (i think). is there any definitive guides or are the usual recommendations (udemy, pluralsight, etc) similar?

or are these docs just what i should be focusing on? https://doc.rust-lang.org/1.0.0-alpha/book/README.html

e: sorry i should also add javascript and python are my strongest languages, so i'm not coming in fresh

# ? Feb 4, 2023 01:00

gonadic io: Feb 16, 2011; >>=

The official book is the usual recommended place to start I see. Feel free to ask questions here of course. None of this helps you spend the money only the time.

# ? Feb 4, 2023 01:18

crazypenguin: Mar 9, 2005; nothing witty here, move along

Also be suspicious of urls with something like "1.0.0-alpha" in them.

https://doc.rust-lang.org/book/

# ? Feb 4, 2023 01:53

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

I've always found it best to learn a language starting with the initial release and then redoing my projects feature-for-feature in each release after.

# ? Feb 4, 2023 01:59

Dominoes: Sep 20, 2007

crazypenguin posted:

Also be suspicious of urls with something like "1.0.0-alpha" in them.

https://doc.rust-lang.org/book/

Rust uses 0-based Versioning, so this about as mature as you can expect.

# ? Feb 4, 2023 05:07

Hadlock: Nov 9, 2004

So I have a pixmap from a renderer in RBGA and need to convert it to a semi proprietary frame buffer format which is effectively 0RBG. I think their plan is to someday support Alpha channel and it would become ARGB. These are all u32

https://en.m.wikipedia.org/wiki/RGBA_color_model

Anyways how would I go about doing that? Some kind of bit shift, read in RGBA, mask for RGB, then create a new u32 starting with 00000000 and then dump RGB at the end?

RRRRRRRRGGGGGGGGBBBBBBBBAAAAAAAA -> 00000000RRRRRRRRGGGGGGGGBBBBBBBB

I'm probably overthinking this but a 1080p display would need this operation done ~2 million times per frame, and then 60 frames per second, on top of the render time. I guess that's a good use of parallelization? How the gently caress do I do a bit shift, or am I even using the right phrase/asking the right question

Edit; this might be a safe crate for that? https://docs.rs/bitfrob/latest/bitfrob/

Safe variant of bytemuck crate https://docs.rs/bytemuck/latest/bytemuck/

Kind of looks like someone ran into the same problem as me

https://users.rust-lang.org/t/display-raw-image-data-in-a-window/74714/7

They/I am using softbuffer because it doesn't use a GPU at all, because reasons

Hadlock fucked around with this message at 12:26 on Feb 7, 2023

# ? Feb 7, 2023 12:11

gonadic io: Feb 16, 2011; >>=

Just `x >> 8` does that, no? Rust u32's shr fills with zeroes.

# ? Feb 7, 2023 12:41

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

If you write the straightforward "loop over every pixel in the framebuffer and replace it with the transformed version" code, Rust will auto-vectorize it for you if you turn on the compiler optimizations.

If that's still not fast enough I would take the ten seconds to figure out how to set up a GPU context before spending minutes-to-hours trying to parallelize and optimize the inherently slow option further.

# ? Feb 7, 2023 13:22

Threep: Apr 1, 2006; It's kind of a long story.

gonadic io posted:

Just `x >> 8` does that, no? Rust u32's shr fills with zeroes.

You can also do this which compiles down to the same shift:

code:

    let [r, g, b, _a] = rgba.to_be_bytes();
    u32::from_be_bytes([0, r, g, b])

# ? Feb 8, 2023 02:29

danishcake: Aug 15, 2004

I'm trying to use an enum to express advancing knowledge about a thing, but some of the knowledge I have is non-copyable (or at least would require careful, unsafe code to copy), and it's making the borrow checker sad.

Trivial example of the rough structure I'm trying to use:

code:

/// The information we know in Stage1
/// While trivial here, my real data cannot be copiable
struct Stage1Contents {
    data: u8,
}

/// The incremental information we know in Stage2, in addition to that known in Stage1
struct Stage2Contents {
    data: u8,
}

/// Our current knowledge of the system
enum IncrementalKnowledge {
    /// Nothing known
    Nothing,

    /// Stage1 has partial knowledge of the system
    Stage1 {
        stage1: Stage1Contents,
    },

    /// Stage2 has full knowledge of the system
    Stage2 {
        stage1: Stage1Contents,
        stage2: Stage2Contents,
    },
}

struct State {
    pub knowledge: IncrementalKnowledge,
}

impl State {
    /// Constructs a new State object
    pub fn new() -> Self {
        State {
            knowledge: IncrementalKnowledge::Nothing,
        }
    }

    /// Advances the state, as new data becomes available
    /// Nothing -> Stage1
    /// Stage1  -> Stage2
    /// Stage2  -> Nothing
    pub fn update(&mut self) {
        self.knowledge = match &self.knowledge {
            IncrementalKnowledge::Nothing => IncrementalKnowledge::Stage1 {
                stage1: Stage1Contents { data: 0 },
            },
            IncrementalKnowledge::Stage1 { stage1 } => IncrementalKnowledge::Stage2 {
                // cannot move out of `*stage1` which is behind a shared reference
                // move occurs because `*stage1` has type `Stage1Contents`, which does not implement the `Copy` trait
                stage1: *stage1,
                stage2: Stage2Contents { data: 0 },
            },
            IncrementalKnowledge::Stage2 { .. } => IncrementalKnowledge::Nothing,
        }
    }
}

fn main() {
    let mut state = State::new();

    state.update();
    state.update();
    state.update();
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=36af80f0d868ceffa2650b4638b42c3b

Am I on a hiding to nothing with this? Any suggestions on how I should restructure the code to avoid it?

# ? Feb 22, 2023 08:38

kujeger: Feb 19, 2004; OH YES HA HA

You could make State.update consume self:

code:

    pub fn update(self) -> Self {
        Self {
            knowledge: match self.knowledge {
                IncrementalKnowledge::Nothing => IncrementalKnowledge::Stage1 {
                    stage1: Stage1Contents { data: 0 },
                },
                IncrementalKnowledge::Stage1 { stage1 } => IncrementalKnowledge::Stage2 {
                    stage1: stage1,
                    stage2: Stage2Contents { data: 0 },
                },
                IncrementalKnowledge::Stage2 { .. } => IncrementalKnowledge::Nothing,
            },
        }
    }

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=3d907d82a9846c10a87311665d1f3fc7

# ? Feb 22, 2023 09:00

gonadic io: Feb 16, 2011; >>=

Another, probably worse, option would be to have your enums contain an Option and use the `take()` method on them when updating state by mut ref

# ? Feb 22, 2023 09:51

kujeger: Feb 19, 2004; OH YES HA HA

You can also wrap the stage contents in Rc/Arc, allowing you to clone the handle but not the content itself, e.g.

code:

struct Stage1Contents {
  data: u8,
}
[..]
enum IncrementalKnowledge {
  Stage1 {
      stage1: Rc<Stage1Contents>,
  },
}
[..]

          IncrementalKnowledge::Stage1 { stage1 } => IncrementalKnowledge::Stage2 {
              stage1: stage1.clone(),
              stage2: Rc::new(Stage2Contents { data: 0 }),
          },

kujeger fucked around with this message at 10:54 on Feb 22, 2023

# ? Feb 22, 2023 10:52

Lime: Jul 20, 2004

Since IncrementalKnowledge has a good "nothing" state, you can mem::replace self.knowledge with that and then move from the previous value you get back:

Rust code:

pub fn update(&mut self) {
    use IncrementalKnowledge::*;
    self.knowledge = match mem::replace(&mut self.knowledge, Nothing) {
        Nothing => Stage1 {
            stage1: Stage1Contents { data: 0 },
        },
        Stage1 { stage1 } => Stage2 {
            stage1: stage1,
            stage2: Stage2Contents { data: 0 },
        },
        Stage2 { .. } => Nothing,
    }
}

# ? Feb 22, 2023 16:59

danishcake: Aug 15, 2004

Thanks for several decent answers. I went for the mem::replace route, and it works beautifully. I have to say, while Rust is generally rather swish, there are distinctly poor language ergonomics in this case.

# ? Feb 23, 2023 08:23

gonadic io: Feb 16, 2011; >>=

What the gently caress is a borrow checker and why is it yelling at me

# ? Feb 23, 2023 08:52

kujeger: Feb 19, 2004; OH YES HA HA

Had somehow missed out on mem::replace, very nice!

# ? Feb 23, 2023 14:53

crazypenguin: Mar 9, 2005; nothing witty here, move along

When you own a type, Rust lets you get away with "partial moves." This is why changing it from `&mut self` to `self` would work. When it's fully-owned, you can track what parts are or are not valid memory, and get all the details right.

When it's not owned, but an exclusive reference (`&mut`), in principle you can do the same tracking. BUT, now the object continues to exist (instead of being de-allocated) if the function returns. This means there's a problem when you have panics and returns happen in unexpected places. If Rust did allow partial moves on an `&mut` value, you'd have to get the equivalent of C++ exception safety right. Which is to say, the rules are:

1. You're hosed.
2. ABANDON ALL HOPE

This is part of why things like `mem::replace` and `mem::take` exist. You can "swap" out something you have an exclusive mutable reference to, and now you own it and can do whatever owner things you want. You just need some valid placeholder to leave in its place, like Indiana Jones's bag of sand.

Every time you use these you should think for a moment about "unwind safety." Most of the time, this doesn't matter, so there's nothing to do! Rust is forcing you to ensure no memory safety errors occur here, but that doesn't mean some other error could happen if you panic in between `take` and putting the intended value back. But it only really matters if you plan on catching panics and continuing, which is relatively rare. (If you do need unwind safety here, you can resolve some of these problems through the trick of creating a 'holder' object that does the initial `replace` and does a `replace` back as part of its `drop` implementation. That way, panic unwinding (which runs drops) will still put back the intended value instead of the placeholder. You'll find this pattern in the `std::Vec` implementation in some places.)

# ? Feb 23, 2023 18:23

Armauk: Jun 23, 2021

I'm working with the sqlx library, and as part of the sqlx::query_as method need to provide a struct -- in this example, Item -- for parsing. I'm dealing with two structs, both results from different queries:

code:

struct Item {
   id: i8,
   name: &str
}

struct ItemWithAge {
   id: i8,
   name: &str,
   age: i8,
}

I can't figure out how I can write this function to accept two other variations of the Item struct without needing to duplicate the code just to switch the struct passed to query_as. Generics in C++ seems to fix this, but Rust doesn't seem to have that implementation.

code:

pub async fn query(db: &Pool<Sqlite>, query: &str) -> Result<Vec<Item>, ()> {
    let results = sqlx::query_as::<_, Item>(query).fetch_all(db).await;
     Ok(results.unwrap())
}

Any suggestions?

Armauk fucked around with this message at 04:01 on Mar 5, 2023

# ? Mar 4, 2023 20:36

VikingofRock: Aug 24, 2008

Could you just implement a ToItem trait (or AsItem, if you are returning a borrowed "view") for all of the things that you want to convert to an item? Then you could either take a ToItem in you generic function, or you could call to_item() ahead of time. (playground link)

# ? Mar 4, 2023 21:19

gonadic io: Feb 16, 2011; >>=

The short answer is that this works:

code:

pub async fn query_generic<MyItem>(db: &Pool<Sqlite>, query: &str) -> Result<Vec<MyItem>, ()>
where
    MyItem: for<'a> FromRow<'a, SqliteRow> + Send + Unpin,
{
    let results = sqlx::query_as::<_, MyItem>(query).fetch_all(db).await;
    Ok(results.unwrap())
}

The long answer (i.e. how you'd get to this) is I first tried just doing (based on the docs to sqlx saying the constraint that query_as uses is the FromRow trait):

code:

pub async fn query_generic<MyItem: FromRow>(db: &Pool<Sqlite>, query: &str) -> Result<Vec<MyItem>, ()>
{
    let results = sqlx::query_as::<_, MyItem>(query).fetch_all(db).await;
    Ok(results.unwrap())
}

But the type errors told me it needs to have the for <'a> existential lifetime and there was another type parameter, and I used cargo doc to look at the structs and see under what conditions they implemented FromRow - seeing that it needed a specific row impl depending on the database.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c6f606e378c7b5a5bd0e1a78a6d5cc19

e: using local cargo doc to inspect your struts and see what impls the derives actually created for them, and with what bounds, is a pro-rear end advanced move btw

E2: personally if a coworker submitted this I would be advocating for them to remove it and just write the line out twice. It's not like it's really saving you much code at all on reflection

gonadic io fucked around with this message at 01:02 on Mar 5, 2023

# ? Mar 4, 2023 21:34

Ranzear: Jul 25, 2013

Just duplicate the damned code, lol. That's exactly what the compiler does with generics anyway.

# ? Mar 5, 2023 12:53

Sagacity: May 2, 2003; Hopefully my epitaph will be funnier than my custom title.

Ranzear posted:

Just duplicate the damned code, lol. That's exactly what the compiler does with generics anyway.

Golang developer spotted

# ? Mar 5, 2023 14:00

Ranzear: Jul 25, 2013

I wish I had such (any) tolerance for golang.

Rust compiler: "This is wrong because you're stupid."
Golang compiler: "This is wrong because I'm stupid."

Edit: I held back my phone posting to double check, and Go still doesn't inline any function with generics. 'Just duplicate the code, dingus' is the more performant solution in Go. If that was the joke then I am now twice as offended by the insinuation.

Ranzear fucked around with this message at 05:19 on Mar 6, 2023

# ? Mar 5, 2023 23:44

prom candy: Dec 16, 2005; Only I may dance

I'm reading the Rust Programming Language and I wrote a really simple shell script in Rust for my first little toy attempt. Rust seems to have so many philosophies that make so much sense to me. My "home" language is Ruby and I'm so sick of all the different ways it lets you gently caress yourself over to satisfy its core philosophies of staying out of your way and being a little cutie pie. Every other page in the Rust book is like "hey remember that one lovely bug you dealt with last week? Rust doesn't let you do that, because why would it?"

I know there's a ton of hype around Rust right now and every new tool seems great to me when I'm in the "building toys" stage. What are the bad parts in Rust?

# ? Apr 7, 2023 18:34

gonadic io: Feb 16, 2011; >>=

Core team take 5 years to do anything, async is still a fractured ecosystem, borrow checker is annoying and still blocks patterns you might want, compiler is slow, uh

# ? Apr 7, 2023 18:47

YOUR UNCOOL NIECE: May 6, 2007; Kanga-Rat Murder Society

Ecosystem problems are what I run in to. Old crates, bad crates, dumb crates, duplicate crates etc.

Once everything is good it's great, but sometimes its a balancing act and annoying. (And hopefully it stays good and your dependencies continue to update and be supported)

# ? Apr 7, 2023 19:55

lifg: Dec 4, 2000; <this tag left blank>; Muldoon

Advice on stack overflow is often invalid if it�s more than a couple years old. Library documentation isn�t great, and sometimes there�s just no one (publicly) using the function you�re trying to figure out.

# ? Apr 7, 2023 20:17

Ranzear: Jul 25, 2013

YOUR UNCOOL NIECE posted:

Old crates, bad crates, dumb crates, duplicate crates

And then there's an async implementation of each, and an async runtime that also falls into each.

# ? Apr 8, 2023 09:55

gonadic io: Feb 16, 2011; >>=

oh speaking of, new rust twitter drama hit this morning https://docs.google.com/document/d/1ErZlwz9bbSI43dNo-rgQdkovm2h5ycuW220mWSOAuok/edit

# ? Apr 8, 2023 10:29

Hadlock: Nov 9, 2004

I dumped a bunch of time and effort into a 2.5D graphics system for a personal project, then the team running the graphics API changed the underlying graphics system it was using (which is fine, that's what pinned versions are for), and the graphics system made some weird breaking change and stopped distributing the pinned version my API was using. I still haven't had a chance to port everything across yet

Anyways TL;DR there's a ton of poorly supported or just plain broken crates out there, and yeah strong agree half the advice on stack overflow is out of date or wrong

Does rust avoid many foot guns? Yes. Are more than half the crates broken or unsupported? Yes. Does it have atrocious video and sound support? Absolutely.

# ? Apr 9, 2023 00:18

tazjin: Jul 24, 2015

gonadic io posted:

oh speaking of, new rust twitter drama hit this morning https://docs.google.com/document/d/1ErZlwz9bbSI43dNo-rgQdkovm2h5ycuW220mWSOAuok/edit

That drama has been going on for a while now.

Looks like what they landed on is less drastic than initially proposed (there's been some backlash already). The restriction on using "rust" in crate names is stupid, but good luck enforcing that.

The only thing I can think of that's directly affected by this as it stands is RustCon Moscow, which I don't think has "official" permission from "The Project(tm) (c)", but good luck enforcing that copyright.

Ranzear posted:

And then there's an async implementation of each, and an async runtime that also falls into each.

Rust life is much better if you just never touch the standard async ecosystem.

# ? Apr 13, 2023 18:40

Vanadium: Jan 8, 2005

It sounds like the core motivation is being able to dunk on people who try to sell lovely off-brand compilers introducing incompatibilities, but they haven't figured out how to get the lawyers to write a policy that a) achieves that b) doesn't piss off the kind of person who lives in an open source community, and I guess it's unclear if that's possible.

Not letting local user groups or meetups use rust in the name seems particularly annoying compared to what other projects do (tho i get protecting rustcon specifically), and also not allowing rust in crate names. I do enjoy that they claim the trademark for clippy at least.

I'm pretty sure at least as it stands if you email the people working on the trademark thing with your proposed not-evidently-hostile use they're just going to be like "yeah that sounds cool go for it", but I get people being nervous about long term.

# ? Apr 13, 2023 22:14

Adbot: ADBOT LOVES YOU

# ? Apr 27, 2024 01:12

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

I'm our company's rep to the Rust Foundation, and I've got some miles on me from other open source trademark kerfuffles, so I have some time set aside next week to write up an opinion to send them. I understand their motivation for sure, and I think a robust trademark policy is important for protecting users. We had to use the Firefox trademark to go after places that packaged malware or sold support scams, and it would have been a lot harder to get prompt action without the registrations and policy we had. Avoiding "I installed Rust and it added a cryptominer"/"I bought a Rust T-shirt and it gave me a rash" sorts of things is important for a bunch of reasons.

I'm surprised that use by user groups and meetups isn't just nominative use. It's a user group and it's about Rust and it's in Miami: it's a Miami Rust� User's Group. Python's policy isn't explicit about user groups, but they seem to be around. (It is explicit about recolourings of the Python logo, though.)

Nick Cameron feels that Rust should be a generic term, like HTML or C++, and that the Rust Foundation/Project should trademark things like "rustc", but I don't think that really provides meaningful protection to the end user, especially since any interoperating compiler will pretty much have to at least offer an option to be invoked as "rustc".

The rust-in-crate-names restriction could have been entirely feasible if at the beginning of crates.io/cargo they'd said "the prefix rust- is reserved for use by the Rust Project or as approved by them blah blah". At this point it would be pretty disruptive to impose I think.

Ultimately, though, I think that the Foundation is going to end up leaving open some avenues of abuse because there's no good legal structure that forbids them without overly constraining good faith uses in ways that an approval mechanism won't be able to scale with. They'll need to use other tools to deal with those abuses, and I can see why it's very appealing to try and solve as many problems as possible with the trademark policy.

# ? Apr 13, 2023 22:34

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »