Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Mata
Dec 23, 2003
Thanks for the tips, I think into boxed slice is what feels best for me:
Rust code:
    let vec = Vec::with_capacity(LARGE_NUM);
    // todo fill
    let arr: Box<[f32; LARGE_NUM]> = vec.into_boxed_slice().try_into().unwrap();
While messing around with this, I keep wishing there was a way to index into [T; 256] arrays using u8 indices. Not for perf reasons, I can imagine it's slower than just using the machine's word size. Not really for safety either, just for that feeling of satisfaction when all the pieces fit together just right.

Adbot
ADBOT LOVES YOU

fart simpson
Jul 2, 2005

DEATH TO AMERICA
:xickos:

Mata posted:

Thanks for the tips, I think into boxed slice is what feels best for me:
Rust code:
    let vec = Vec::with_capacity(LARGE_NUM);
    // todo fill
    let arr: Box<[f32; LARGE_NUM]> = vec.into_boxed_slice().try_into().unwrap();
While messing around with this, I keep wishing there was a way to index into [T; 256] arrays using u8 indices. Not for perf reasons, I can imagine it's slower than just using the machine's word size. Not really for safety either, just for that feeling of satisfaction when all the pieces fit together just right.

solution: use bigger arrays and just pad the end with zeroes

gonadic io
Feb 16, 2011

>>=
You'll get used to the endless `as usize` eventually, sorry.

gonadic io
Feb 16, 2011

>>=
Maybe one day we'll get `n: usize where n <= 256` but alas, not today

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
How do I convert a String in Rust to go into a C function call requiring wchar_t*? For what it's worth, the function involved just sees English alphanumeric characters, but I think it was made wchar_t* for futureproofing. I don't have the leeway to change the API.

gonadic io
Feb 16, 2011

>>=
The most common library seems to be https://docs.rs/widestring/latest/widestring/index.html

If you really don't want to use a library (I strongly recommend you do so) you can use a bunch of #[cfg(target_os= ...)] annotations to have one u16 mod and one u32 mod, that's the more general solution to APIs that have different impls on different targets.

gonadic io fucked around with this message at 19:13 on May 26, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
The library's fine. I don't really know what's out there. Now, is there a way to turn a *const c_char into a w_char by going through a widestring? I thought I'd try starting from CStr in an unsafe block but I can't seem to work it from there.

Edit: Ehh I think I found something but wow this is pretty goofy stuff to do! My Rust code is the meat in a C/C++ sandwich so I'm getting hit on both sides.

Edit Edit: No I did not find something. I can't seem to find an incantation that takes me from a C char* coming in to wchar_t* going out. I can dance around a pile of conversions all day but never seem to get the two ends together. My default path here is to just grind the widestring APIs and sketch on paper any possible path.

Rocko Bonaparte fucked around with this message at 19:30 on May 27, 2022

gonadic io
Feb 16, 2011

>>=
after some loving around this was the best I got:
Rust code:
use std::ffi::CStr;
use widestring::{WideCString, WideChar};

fn main() {
    // presumably obtained from a c api somewhere
    let char_ptr = b"abcd\0".as_ptr() as *const i8;

    // does not allocate
    // SAFETY: The caller has provided a pointer that points to a valid C
    // string with a NUL terminator of size less than `isize::MAX`, whose
    // content remain valid and doesn't change for the lifetime of the
    // returned `CStr`.
    let cstr = unsafe { CStr::from_ptr(char_ptr) };

    // does not allocate if `cstr` is all valid utf-8
    let str = cstr.to_string_lossy();

    // does allocate
    let wcstring = WideCString::from_str(str).unwrap();

    println!("{}", wcstring.display());

    // off to the bad place with you
    // Safety
    // The pointer must be returned to Rust and reconstituted using
    // from_raw to be properly deallocated. Specifically, one should not
    // use the standard C free function to deallocate this string.
    // Failure to call from_raw will lead to a memory leak.
    let wchar_ptr: *mut WideChar = wcstring.into_raw();
}
e: if you know the len ahead of time you can replace the `CStr::from_ptr` call with `slice::from_raw_parts` followed by one of the `CStr::from_bytes_*` methods instead

e2: if you're returning from whatever c function uses the wide string back into this rust fn you can do e.g.
Rust code:
// The caller must ensure that the string outlives the pointer this function
// returns, or else it will end up pointing to garbage.
let wchar_ptr: *mut WideChar = wcstring.as_mut_ptr();

// off to the bad place with you
unsafe {my_c_ffi_fn(wchar_ptr);}

// do more rust things
println!("{}", wcstring.display());

// wcstring implicitly deallocated when scope ends
then you don't need to clean up after yourself

gonadic io fucked around with this message at 20:09 on May 27, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
I was close!

I was using WideCString::from instead of WideCString::from_str, and I wasn't using the into_raw() conversion. I would not have recognized from into_raw()'s signature that it produces something compatible with wchar_t*. I guess the UChar type is part of widestring and mates with the u32 type that wchar_t is in my project.

Holy gently caress. Well, I asked for it haha.

gonadic io
Feb 16, 2011

>>=

Rocko Bonaparte posted:

I would not have recognized from into_raw()'s signature that it produces something compatible with wchar_t*. I guess the UChar type is part of widestring and mates with the u32 type that wchar_t is in my project.

assuming you mean the WideChar type then yes, there's a section in the docs about ffi with wchar_t
https://docs.rs/widestring/latest/widestring/index.html#ffi-with-cc-wchar_t

n.b. important to note that should you target windows platforms, WideChar and wchar_t will be 2 bytes, not 4.

e: oh wait I see the confusion. The relevant definitions are:
Rust code:
pub type WideCString = UCString<WideChar>;
pub struct UCString<C: UChar> { /* fields omitted */ }
impl<C: UChar> UCString<C> {
    pub fn into_raw(self) -> *mut C {...}
}
impl UCString<u32> { /* some u32 specific stuff */ }

/// Alias for [`u16`] or [`u32`] depending on platform.
/// Intended to match typical C `wchar_t` size on platform.
#[cfg(not(windows))]
pub type WideChar = u32;

#[cfg(windows)]
pub type WideChar = u16;
`UChar` is a trait that u16 or u32 implement, and is mostly just there because some functions are u32 only, so the function `into_raw` returns *mut to whatever its char type is, which on linux is just u32, so it returns `*mut u32` which is `unsigned long*` i.e. `wchar_t*`

gonadic io fucked around with this message at 20:49 on May 27, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
I'm specifically on Linux but I'm trying not to use the specifically sized stuff anyways. It's just that the compiler likes to throw u32 in my face.

I've got another nasty one. I have this type in C:
code:
const wchar_t* const * const keys
It was automatically wrapped like this:
code:
keys: *const *const wchar_t,
[code]

I'm not really good on my const in Rust yet so I really just don't have any idea how I'd prepare this. I have to declare from scratch and am not working with an existing construct. I tried:

[code]
    let values: Vec<*const u32> = vec![w_json_data.into_raw()];
....
(..., &values as *const *const u32, ...)
And of course "casting `&Vec<*const u32>` as `*const *const u32` is invalid." I'm not surprised but I don't know how I'd do it instead. I really jumped into the deep end putting myself in between getting called by C and then calling C lol.

gonadic io
Feb 16, 2011

>>=
Similarly to before, `Vec` has `as_ptr` and `into_raw_parts` methods depending on whether you want automatic or manual deallocation respectively. You'll find that `as` casting is not used very much on complex types, only on primitives like converting between integer sizes and so on.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
As a general rule, should I be hunting for as_ptr and into_raw_parts functions when I hit a bind like that? I'm guess it's standardized in a trait or something.

tinaun
Jun 9, 2011

                  tell me...
only .from() .into() .try_from() .try_into() and .from_str() are from traits, the rest are just convention. Every smart pointer type in the std has the .as_[mut_]ptr() / .into_raw() methods, though.

Be extra careful with ownership when working with ffi, it’s easy to accidentally drop smart pointers when they go out of scope before the ffi uses them, and you don’t want to free memory allocated in rust from the c functions and vice versa.

gonadic io
Feb 16, 2011

>>=
Rust has a lot of method conventions like that due to its lack of a more expressive type system. Another common one is and_then which is a bespoke non-trait method that's very standardised.

I think the best thing you could have done ctrl+f for *const or *mut on the docs page and see what that gives you. I couldn't get rustdoc's type search working here.

e: when putting together the previous snippet I was mostly just scrolling down to any method starting with from/into/to and seeing what the options were. Looking at your edits you were mostly on the right track by converting to a CStr first

gonadic io fucked around with this message at 10:12 on May 28, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Thanks for all the little tips there. That gives me some street smarts I wouldn't be finding normally in some O'Reilly book.

I kind of picked for work an example of having C calling Rust calling C on the assumption it would really suck, but I expect we'd have to do a lot of this if we started to stir Rust into our work. It's been a great way of quickly ending the Honeymoon period and looking at the language more objectively. It's still holding up and I do intend to keep on using it.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

“Rust for Rustaceans” has a section on FFI and generally covers unsafe and friends better than most of the other Rust books (excluding the ‘nomicon), FWIW.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
With FFI and everything, does a CString directly correspond to a char*? When I ran cbindgen, it used const char* for CString in the function bindings, but it used CString for a struct declaration. I tried to declare that as char*, which I think severely angered it. I get a segfault when I try to assign to one of the CString fields that I declare in the header as char*.

Edit: I guess I should use *mut c_char for the struct fields instead of CString and into_raw() to assign to them. Fun times!

Rocko Bonaparte fucked around with this message at 06:41 on Jun 2, 2022

gonadic io
Feb 16, 2011

>>=
Are you passing the whole struct through the ffi barrier? If yes, you're right and it needs to be a raw pointer in it (and make sure you have repr(C) too). If no, then bindgen made the assumption that when you create and manipulate the struct on the rust side you'd rather not bother with raw pointers so expects you to convert from raw pointer when creating the struct (and back into raw when passing stuff to C).

E: what I mean is, once you're firmly into the rust domain you absolutely do not want to be messing with pointers and want to have converted into a cstring, but anything that goes across the boundary does need to be converted to use pointers first

gonadic io fucked around with this message at 10:11 on Jun 2, 2022

xgalaxy
Jan 27, 2004
i write code
FFI is probably one of the most difficult things to do in Rust and if you are new the language is probably the last place you should be.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Yeah FFI has been a much more surprising pain in the rear end compared to other language interoperations I've used before. However, I think half of my problems have actually just been with strings; they wear me out before I have to deal with the FFI stuff.

I keep getting ahead of where I am in Programming Rust. I stopped after basic syntax originally and then got my rear end handed to me when I got some functions returning Result<>. I think the next chapter was going to be strings in detail and that should hopefully clear some stuff up. It's a lot easier to pay attention to that stuff after it's hit me in the face a few times.

I was originally just going to wait until we have an opportunity for a new standalone application in order to try Rust, but I just kept seeing stuff about people integrating Rust with Python, Rust with userspace C++ stuff, and then Rust with kernelspace C stuff. The Python and kernelspace stuff matter to me in my job. So I just decided, "gently caress it, let's just see what it's like to integrate with it." Hahaha it's not so good that way.

Counterpoint: It's also not fair to compare an interoperating Rust function to its raw C equivalent because the Rust version is doing what the function has to do and it's converting to/from C. Yes, that's a hint that I should have a layer doing the translation and then a "pure" layer underneath. Anyways, it's shown me how difficult it would be to recommend this at a wider level in my team/organization if it has to be done with an existing C/C++ project.

Speaking of C: Is there anything I can particularly do to ensure I can bind with ANSI C code? The cbindgen is technically generating C++, but we have some code that is pure ANSI C, and then there's trying to do kernel-level code with it some day. For that, I figured I'd find what's being said out there about that ongoing project in the kernel, but I thought I'd ask here too.

Rocko Bonaparte fucked around with this message at 20:59 on Jun 2, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Do any of you debug Rust code being called from C applications? I am trying to figure out if I can do this with clion.

Ralith
Jan 12, 2011

I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today
I've never used clion, but debugging arbitrarily mixed rust/C should normally Just Work.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
I guess I'll just try to twist clion's arm to launch the test C binary and step down into the Rust code from there. It wasn't looking straightforward so I figured I'd ask before I broke my brains on it.

So next one: What should my weapon of choice be if I'm trying to expose both C bindings and Python bindings? I could just use ctypes and wrap the C calls, but I'd be creating a serial dependency chain; I anticipate the Python bindings unwittingly falling apart if I end up changing something in the Rust side. I'd rather present both the C and the Python bindings from the Rust code itself so they can change and rebuild with the Rust code changes directly.

gonadic io
Feb 16, 2011

>>=
Clion can also attach to existing processes, so no need to launch everything via it.

I think usually what people use, especially when they've got external tooling, is having a build.rs file that runs on compile and runs whatever gen tools you need. https://doc.rust-lang.org/cargo/reference/build-scripts.html

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
It looks like there's a cpython crate I can just use and since I'm pretty cozy with CPython stuff that I might as well just use it.

Something that might be more interesting: I wanted to do some paranoia one some of these unwraps() I was using on various text types in case there were some zeroes in the stream. I don't think they'd necessarily hit in all cases ever, but I tried to cover them just to be thorough and not just have all these unwrap calls running naked around the code. Now they're tucked in helpers in a way that they should never actually hit. The problem is that these functions are oh-so-similar yet I have to write a different one for each of the types. Can this all just be done better? Can I consolidate this?

(note I'm stuck with Rust 2018)

code:
fn make_cstring_from_str(in_str: &str) -> CString {
    let remapped: String = in_str.chars()
        .map(|x| match x {
            '\u{0}' => ' ',
            _ => x
        }).collect();

    CString::new(remapped).unwrap()
}

fn make_cstring_from_string(in_str: String) -> CString {
    let remapped: String = in_str.chars()
        .map(|x| match x {
            '\u{0}' => ' ',
            _ => x
        }).collect();

    CString::new(remapped).unwrap()
}

fn make_cstring_from_u8(in_ar: &[u8]) -> CString {
    let remapped: String = in_ar.to_vec().iter()
        .map(|x| match x {
            0 => ' ',
            _ => *x as char
        }).collect();

    CString::new(remapped).unwrap()
}

gonadic io
Feb 16, 2011

>>=
good news,

https://doc.rust-lang.org/std/ffi/struct.CString.html#creating-a-cstring posted:

The CString::new method will actually check that the provided &[u8] does not have 0 bytes in the middle, and return an error if it finds one.

e: see also `CStr::from_bytes_until_nul`

given your requirements I think erroring on 0s should be fine, it beats unwraps and you can log errors properly, did you specifically want the silent replacement behaviour? `CString::new` does it with memchr even so will be extremely fast compared to anything vaguely idiomatic.

gonadic io fucked around with this message at 10:18 on Jun 8, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Yeah I saw what the error type was telling me and decided I'd want to just replace the characters if I got a surprise 0.

I haven't run enough Rust code to be certain of how I'd get bit by stuff like this, but one of the conversions I do will come from process stdout and that's been a huge bag of bear traps for me in other worlds. I'm using some of that output to create a hash and I'm fine with soldiering on if I get a surprise zero.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Are there any mechanisms in place for standard crates from crates.io to independently download source and build them? We'd like to incorporate Rust but I have a corporate requirement to independently build source and run some security scanning in between.

gonadic io
Feb 16, 2011

>>=
Sure yeah some options here: https://stackoverflow.com/questions/32267233/how-to-build-a-project-using-cargo-in-an-offline-environment

easiest is using `cargo vendor` for your app but that's limited (although sounds like enough for your use case) so there's also more general offline crates.io local mirror tools etc

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
So that's going to be an ongoing thing. I found out about Software Build of Materials at the US Open Source Summit and supply chain security so I suspect this kind of thing is going to become much easier soon.

A new thing: I'm trying to do a command-line utility and I'm doing command line parsing. I picked up the clap crate and I'm surprised with a lot of the logic I have to write. A particular situation is subcommands ie the "pull" in "git pull." It looks like I have to do a conditional to find if each command was set--or perhaps I should say I have to do a match block. I've seen implementations in other languages just use callbacks for that kind of thing. Is that available in clap or some other command-line parsing library in the Rust ecosystem?

crazypenguin
Mar 9, 2005
nothing witty here, move along
With clap’s derive feature, it should just be an enum.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
It looks like Args had an action field associated with them and that made we think I could trigger code on arguments as well as subcommands, but subcommands did not have an action field. I am guessing the Arg action is just a more advanced routine you can insert to process the input (I guess like Python's argparse?)

I was editing some code in the middle of my file and got plagued by this semi-classic:
code:
error: cannot satisfy dependencies so `std` only shows up once
  |
  = help: having upstream crates all available in one format will likely make this go away
My project.toml is targeting dylib. I see a recommendation to use -C prefer-dynamic with rustc, but what about at the project level? I'm trying to build with cargo. I'm also surprised I got whacked by this now. I have lib and bin paths defined, and was working on both and running the resulting application multiple times before this started happening. It kept on happening even after cleaning the project.

Edit: I tried to add this to my cargo.toml and it didn't make a difference:
code:
[build]
rustflags = ["-C", "prefer-dynamic"]
Edit, what the gently caress I changed my lib type from dylib to cdylib and it was happy.

Rocko Bonaparte fucked around with this message at 23:34 on Jul 14, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
I got more build goofiness. I am betting I can't really get any help unless I posted literally everything and I can't really do that.

I have a src/bin/application.rs and a src/lib/lib.rs file. The lib.rs file has some FFI references to a C library. I can see in the build process that it builds correctly and includes the library. However, my application.rs immediately started to fail with missing references to that library in lib.rs the moment it started making calls to the functions in lib.rs that are using them.

If I run cargo build --verbose, I can see that application.rs is not being build with the library reference I need, so I'm not surprised it's missing it. The reference is in my build.rs. It's being set with rustc-link-lib. The lib.rs file seems to get it. What I gather from documentation is that this is supposed too only be set for stuff in a lib folder and not for a bin folder. So I guess the build system thinks its doing what it thinks it should do, but what am I supposed to do with my application calling into lib.rs?

I guess my problem is that lib.rs's functions must be getting inlined or something? How would I properly instead set the application to invoke stuff out of lib? I am import my lib code using:

code:
#[path = "../lib/lib.rs"] mod lib;
So I think I am basically inlining it. So what should I be doing instead?

Vanadium
Jan 8, 2005

You shouldn't need that mod item, you should be able to refer to the lib module by yourcratenamehere::lib by default. Maybe that makes things better?

If you can't post the whole thing to get help, maybe you can recreate the problem in a minimal example repo? It doesn't sound like there are too many moving parts, except maybe a sample external C library.

Vanadium fucked around with this message at 20:17 on Jul 16, 2022

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
I'll post better details tomorrow. I managed to get it to build after some hacking. There is still a problem of the binary build ignoring rustc-link-lib; I had to explicitly set the -l flag in the cargo.toml in order for the linkage to the external library to stick. That's still a sore point for me. The other issues had to do with how I was organizing files and folders. I think the lesson learned so far is that I can't do that willy-nilly. There are some hidden assumptions for one, and I don't know all the details on how it works even in general without the assumptions. That's my problem.

I guess I should say that the next chapter I had to read in the Programming Rust book was about modules hahaha.

Ralith
Jan 12, 2011

I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Rocko Bonaparte posted:

code:
#[path = "../lib/lib.rs"] mod lib;

This, specifically, is the cause of your problem. You almost never need to use #[path]. By applying it like this, you're making your use of that code independent of your library crate, so the rustc-link-lib information from your library's build.rs does you no good. Just use your library crate as a crate, like Vanadium said, and everything will Just Work.

Rocko Bonaparte posted:

I had to explicitly set the -l flag in the cargo.toml in order for the linkage to the external library to stick.
Do not do this.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Okay I'll post more here. This isn't a call to any kind of action or request of advice necessarily but me venting about stomping on, like, every rake that the module and build system has.

I had originally structured my project something like this:
src/
src/lib/
src/lib/lib.rs
src/bin/
src/bin/
src/bin/mainapp.rs
build.rs
Cargo.toml
[/code]

I wound up with:
code:
src/
src/lib.rs
src/mainapp.rs
build.rs
Cargo.toml
In the original structure, I couldn't figure out how to get mainapp.rs to find code from lib.rs without that #[path = butt] stuff at the top of the file. I still don't know how I would have done it. So I ended up just having the file cohabitate. There were additional problems. I didn't wrap lib.rs's code in a pub mod declaration to wrap it up in a module and namespace or whatever Rust calls it. So I was pretty much doomed from the start. Like, maybe I should just try it again now that I have one valid solution, but maybe I should just read the chapter on modules first.

What I couldn't fix without a ham-fisted solution was linking mainapp's binary output with an external C-based library. In my build.rs, I had added a cargo:rustc-link-lib println for it and that was working great for the library that lib.rs created. When it came time to link the console application from mainapp.rs, cargo wouldn't bother to add that library to the rustc command line. I then saw in the documentation that cargo:rustc-link-lib by design only applies to library builds and not for binary builds. I ended up just having to add a cargo:rustc-link-arg line specifically adding the -l flag for it in my build.rs. Not impressed.

Question: Why do I use [lib] for libraries in Cargo.toml but have to use [[bin]] for binaries?

Vanadium
Jan 8, 2005

A package can have multiple binary crates in it but only one library crate, and idk I guess [[bin]] is toml for I'm just one entry in a list of potentially many.

fwiw something about the module system trips me up every time. I don't wanna say it's bad or even less intuitive than other languages, it just makes me go "... huh." surprisingly often.

Edit: And yeah, I still (without evidence/authoritative amounts of experience/having tried it) is that your problem is that the binary crate doesn't use the library crate, but instead incorporates its source independently.

Vanadium fucked around with this message at 04:43 on Jul 20, 2022

Adbot
ADBOT LOVES YOU

Ralith
Jan 12, 2011

I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Rocko Bonaparte posted:

I ended up just having to add a cargo:rustc-link-arg line specifically adding the -l flag for it in my build.rs. Not impressed.

As explained above, this feels bad because it's the wrong way to do it. Follow the advice given in Vanadium's earlier post.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply