Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
pseudorandom name
May 6, 2007

You'll want to use the binary stream IO functions (fread and fwrite) to do binary IO, not the text functions like fprintf.

Adbot
ADBOT LOVES YOU

roomforthetuna
Mar 22, 2005

I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!
Also, if you're going to be writing structures to files you definitely don't want to use the endian-safe per-byte thing, because that won't write the bytes out in the order in which they occur!

Clobbersaurus
Feb 26, 2004

Does one of you know how pairs work? I'm totally clueless.

I'm working on a program that needs to be fast and memory efficient, and I recently decided that the value in a map I have should be 2 ints instead of one. Using pairs jumps out at me, because the key/value is also a pair. So each map entry would be,

pair<char*, pair<int,int>>.

In short, I'd really like to avoid using an int* here, because allocating them dynamically on the heap would be really inefficient, and I don't want to deal with block allocation. What would be ideal is just an 8-byte stack object, like a uint_64 (that I could just byte shift, etc), but I want this code to compile on 32bit machines, so that rules that out.

I guess my fundamental question is - what exactly is the memory involved in a pair? Are pairs pointers that need to be stored, or some compiler trickery (that somehow lumps them together)? What would using pairs cost me, in terms of bytes, per map entry? I've had a really hard time finding reference material for this, so sorry for the simple questions.

e: spelling edit

Clobbersaurus fucked around with this message at 21:26 on May 4, 2011

floWenoL
Oct 23, 2002

Clobbersaurus posted:

I guess my fundamental question is - what exactly is the memory involved in a pair? Are pairs pointers that need to be stored, or some compiler trickery (that somehow lumps them together)? What would using pairs cost me, in terms of bytes, per map entry? I've had a really hard time finding reference material for this, so sorry for the simple questions.

Basically, "compiler trickery". A pair is essentually struct { T1 first; T2 second; }.

Brecht
Nov 7, 2009

Clobbersaurus posted:

I'm working on a program that needs to be fast and memory efficient, and I recently decided that the value in a map I have should be 2 ints instead of one. Using pairs jumps out at me, because the key/value is also a pair. So each map entry would be,

pair<char*, pair<int,int>>
Why not
code:
struct my_two_ints { int height; int weight; };
std::map<std::string, my_two_ints> clobber_map;
? What does the pair get you that a struct doesn't? And why are you keying on a char* and not a std::string?

quote:

I guess my fundamental question is - what exactly is the memory involved in a pair?
Don't worry about it, unless you know for a fact you need to worry about it. Every container in the STL is as efficient as is reasonably possible by default.

Write your program to best model your problem. Use the things that C++ gives you, and don't feel like you need to do better than the obvious solution until you can prove the obvious solution is inadequate. It almost never is.

Brecht fucked around with this message at 23:08 on May 4, 2011

A MIRACLE
Sep 17, 2007

All right. It's Saturday night; I have no date, a two-liter bottle of Shasta and my all-Rush mix-tape... Let's rock.

Doesn't a map<typeA,typeB> make a pair<typeA,typeB>? So clobber_map entries are pair<string,my_struct>?

Clobbersaurus
Feb 26, 2004

floWenoL posted:

Basically, "compiler trickery". A pair is essentually struct { T1 first; T2 second; }.

Thanks - I'll move ahead with the pair and check the memory when I'm finished - by the sound of it, it'll just take 4 extra bytes per entry, and the pair just keeps them together in memory.

Brecht posted:

Why not
code:
struct my_two_ints { int height; int weight; };
std::map<std::string, my_two_ints> clobber_map;
? What does the pair get you that a struct doesn't? And why are you keying on a char* and not a std::string?

Don't worry about it, unless you know for a fact you need to worry about it. Every container in the STL is as efficient as is reasonably possible by default.

Write your program to best model your problem. Use the things that C++ gives you, and don't feel like you need to do better than the obvious solution until you can prove the obvious solution is inadequate. It almost never is.

When I said I needed it to be memory efficient, I meant I'll have on the order of a hundred million entries in this map. If I can avoid having an extra pointer it'll save 400 megabytes of memory, so I do care about the gritty details. I agree with you in principle though. By the gist of floWenoL's post, the struct and pair should be equivalent (and hopefully I can deal with pairs on the stack - not sure if this is possible with structs).

A MIRACLE posted:

Doesn't a map<typeA,typeB> make a pair<typeA,typeB>? So clobber_map entries are pair<string,my_struct>?

Basically you create the pair, and then give it to the map (if that's your question). In his example you're right, except "my_struct" should be "my_two_ints". It's neat because when you access the entries you get the pair back, so you can get the key or value.

That Turkey Story
Mar 30, 2003

Clobbersaurus posted:

When I said I needed it to be memory efficient, I meant I'll have on the order of a hundred million entries in this map. If I can avoid having an extra pointer it'll save 400 megabytes of memory, so I do care about the gritty details. I agree with you in principle though. By the gist of floWenoL's post, the struct and pair should be equivalent (and hopefully I can deal with pairs on the stack - not sure if this is possible with structs).

Yes, you can put an instance of any type on the stack in C++ (of course, that type itself might internally allocate memory on the free store if it wants, i.e. an std::map or std::string).

Sorry to be that guy, but even though you haven't given us much information you are pretty much definitely going about whatever it is that you want to do in the wrong way. Your scenario already implies occupying over 2 gigs of memory for storing this data alone. I highly doubt you actually need hundreds of millions of elements all in memory at once. If you're in 32-bit land, as it sounds, you're realistically running out of address space already. Give us some more information and we can probably tell you a different way to approach your problem since explicitly representing hundreds of millions of objects in a map, or really any datastructure in memory, rarely makes sense.

Brecht
Nov 7, 2009
This is an interesting aspect of C++... it's mature enough that when you begin to approach its limits, in a "seems weird I have to think about stuff in this way" kind of way, the problem is effectively always on your end.

pr0metheus
Dec 5, 2010
You could use a hash_map, or an array (which would require changing logic of your program most likely a bit). That should reduce the amount of overall memory needed.

And yea, you should explain your problem to us so we can suggest a better algorithm/data-structure.

pr0metheus fucked around with this message at 18:07 on May 5, 2011

raminasi
Jan 25, 2005

a last drink with no ice
I need to get the first three objects from a sequence but I don't have a random access iterator. I don't care what order I get them in.

code:
	auto p = points.points_begin();
	plane = r_plane_3(*(p++), *(p++), *(p++));
1. Will this work?
2. Is there a better way?

nielsm
Jun 1, 2009



I don't think the behaviour of multiple uses of the ++ operator (either prefix or postfix) in a single statement is well-defined.
I'd write a function that takes a non-const reference to your iterator, eats three items from it, and returns your plane.

chiyosdad
May 5, 2004

"I wish I were a bird!"
Hello, I have a program which compiles fine, but at the linking stage I get (several instances of) this error:

code:
main.cpp:(.text+0x3502): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::basic_ofstream(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::_Ios_Openmode)'
I googled it and most people that are getting this seem to be getting it because they're using gcc instead of g++, but I'm already using g++. It sounds like g++ can't find the library where ofstream is defined? I tried this sample program and it compiled and ran fine:

code:
#include <iostream>
#include <fstream>

int main()
{
	std::ofstream lol("test.txt");

	lol << "testing\t" << 57 << std::endl;
	lol.close();

	return 0;
}
So I guess I'm doing something different in my program. But the linker doesn't give a line number, so I don't know what's causing it.

To compile I'm using

code:
g++ -std=c++0x -lstdc++ -o main main.cpp
Thanks in advance.

UraniumAnchor
May 21, 2006

Not a walrus.

nielsm posted:

I don't think the behaviour of multiple uses of the ++ operator (either prefix or postfix) in a single statement is well-defined.
I'd write a function that takes a non-const reference to your iterator, eats three items from it, and returns your plane.

Doesn't the , operator make it a sequence point and therefore well defined?

Edit: Ah, never mind, it's not actually a comma operator in this context:

http://en.wikipedia.org/wiki/Sequence_point (see part 4)

UraniumAnchor fucked around with this message at 00:17 on May 6, 2011

TasteMyHouse
Dec 21, 2006

UraniumAnchor posted:

Doesn't the , operator make it a sequence point and therefore well defined?

the "," in a function parameter list is not the , operator

raminasi
Jan 25, 2005

a last drink with no ice

chiyosdad posted:

Hello, I have a program which compiles fine, but at the linking stage I get (several instances of) this error:

code:
main.cpp:(.text+0x3502): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::basic_ofstream(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::_Ios_Openmode)'
I googled it and most people that are getting this seem to be getting it because they're using gcc instead of g++, but I'm already using g++. It sounds like g++ can't find the library where ofstream is defined? I tried this sample program and it compiled and ran fine:

code:
#include <iostream>
#include <fstream>

int main()
{
	std::ofstream lol("test.txt");

	lol << "testing\t" << 57 << std::endl;
	lol.close();

	return 0;
}
So I guess I'm doing something different in my program. But the linker doesn't give a line number, so I don't know what's causing it.

To compile I'm using

code:
g++ -std=c++0x -lstdc++ -o main main.cpp
Thanks in advance.
Are you trying to pass a const ofstream & somewhere?

raminasi
Jan 25, 2005

a last drink with no ice

UraniumAnchor posted:

Doesn't the , operator make it a sequence point and therefore well defined?

Edit: Ah, never mind, it's not actually a comma operator in this context:

http://en.wikipedia.org/wiki/Sequence_point (see part 4)

Oh, yeah.

quote:

Between the previous and next sequence point an object shall have its stored value modified at most once by the evaluation of an expression. Furthermore, the prior value shall be accessed only to determine the value to be stored.
Technically that's C99 and I'm using C++0x but it still doesn't sound like a good idea.

chiyosdad
May 5, 2004

"I wish I were a bird!"

GrumpyDoctor posted:

Are you trying to pass a const ofstream & somewhere?

No.

Is there a way to figure out where the "main.cpp:(.text+0x3502)" location refers to?

Edit: that doesn't suck. Like I assume it means 0x3502 bytes or characters into the text file? There is no way to count that far...

chiyosdad fucked around with this message at 00:34 on May 6, 2011

pseudorandom name
May 6, 2007

Figure out what the base address of .text is (third column in readelf -s output), add it to 0x3502, run it through addr2line.

chiyosdad
May 5, 2004

"I wish I were a bird!"

pseudorandom name posted:

Figure out what the base address of .text is (third column in readelf -s output), add it to 0x3502, run it through addr2line.

I assume "output" means the .o output from the g++? I get something like this:

Symbol table '.symtab' contains 10493 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS main.cpp
2: 0000000000000000 0 SECTION LOCAL DEFAULT 3419
3: 0000000000000000 0 SECTION LOCAL DEFAULT 3421
4: 0000000000000000 0 SECTION LOCAL DEFAULT 3422
5: 0000000000000000 0 SECTION LOCAL DEFAULT 3423
6: 0000000000000000 0 SECTION LOCAL DEFAULT 3425
7: 0000000000000000 0 SECTION LOCAL DEFAULT 3426
8: 0000000000000000 0 SECTION LOCAL DEFAULT 3427
9: 0000000000000000 0 SECTION LOCAL DEFAULT 3428
10: 0000000000000000 0 SECTION LOCAL DEFAULT 3429
11: 0000000000000000 0 SECTION LOCAL DEFAULT 3431
12: 0000000000000000 0 SECTION LOCAL DEFAULT 3433
13: 0000000000000000 0 SECTION LOCAL DEFAULT 3434
14: 0000000000000000 0 SECTION LOCAL DEFAULT 3435
15: 0000000000000000 0 SECTION LOCAL DEFAULT 3437
16: 0000000000000000 0 SECTION LOCAL DEFAULT 3439
17: 0000000000000090 1 OBJECT LOCAL DEFAULT 3422 _ZStL8__ioinit
18: 0000000000000000 0 SECTION LOCAL DEFAULT 3441
19: 0000000000000000 0 SECTION LOCAL DEFAULT 3442
...

I'm not sure what to do with this?

I tried
code:
 
addr2line -e main.o 0x3502 
and got
code:
main.cpp:0
??

pseudorandom name
May 6, 2007

Sorry, I meant readelf -S, not readelf -s.

chiyosdad
May 5, 2004

"I wish I were a bird!"

pseudorandom name posted:

Sorry, I meant readelf -S, not readelf -s.

Ok, thanks. So the output is

code:
There are 9244 section headers, starting at offset 0xc9488:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
...
  [3309] .text             PROGBITS         0000000000000000  00006834
       0000000000009628  0000000000000000  AX       0     0     4
...
So the offset is 00006834? Is that in decimal? So... 6834+0x3502 = 20404 = 0x4FB4? That gives

addr2line -f -e main.o 0x4FB4
main
main.cpp:0

0x6834 + 0x3502 = 0x9D36 gives

addr2line -f -e main.o 0x9D36
??
??:0

Lol...

pseudorandom name
May 6, 2007

No, the address, which is 0. So 0x3502 is the actual address.

Are you compiling with debugging symbols? You can't turn an address to a line number without them.

raminasi
Jan 25, 2005

a last drink with no ice
How about passing one by value?

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
That constructor (passing a std::string to std::ofstream) was added in C++0x, but the compiled libstdc++ library on your system does not actually support C++0x. Normally this doesn't matter for template classes (which std::ofstream secretly is) because the system will happily just emit it when you use it, but in this case there's an extern template declaration for std::ofstream which causes the compiler to believe that some external translation unit will provide symbols for all of its members. And indeed, there's a corresponding explicit instantiation of std::ofstream in the standard library, except that it (because it was compiled without '0x support) has no idea that it needs to provide that constructor, or even that that constructor exists.

The minimal test case for this would be something like:

code:
#include <fstream>
int main() {
  std::string filename = "welp";
  std::ofstream out(filename);
}
Your reduced test case works fine because std::ofstream provides a const char* constructor in '03, and so your libstdc++ actually exports this symbol.

ETA: The fix is to either pass .c_str() to the ofstream constructor or install a standard library which matches your headers.

rjmccall fucked around with this message at 02:53 on May 6, 2011

chiyosdad
May 5, 2004

"I wish I were a bird!"

rjmccall posted:

That constructor (passing a std::string to std::ofstream) was added in C++0x, but the compiled libstdc++ library on your system does not actually support C++0x. Normally this doesn't matter for template classes (which std::ofstream secretly is) because the system will happily just emit it when you use it, but in this case there's an extern template declaration for std::ofstream which causes the compiler to believe that some external translation unit will provide symbols for all of its members. And indeed, there's a corresponding explicit instantiation of std::ofstream in the standard library, except that it (because it was compiled without '0x support) has no idea that it needs to provide that constructor, or even that that constructor exists.

The minimal test case for this would be something like:

code:
#include <fstream>
int main() {
  std::string filename = "welp";
  std::ofstream out(filename);
}
Your reduced test case works fine because std::ofstream provides a const char* constructor in '03, and so your libstdc++ actually exports this symbol.

ETA: The fix is to either pass .c_str() to the ofstream constructor or install a standard library which matches your headers.

YESSS!!! That was exactly the problem! Thank you so much!!!

I got it to work by ctrl+fing all instances of ofstream and putting in the .c_str(). Kind of amazing that that was the only problem given that the library is retarded.

dizzywhip
Dec 23, 2005

I've got a homework assignment that's driving me nuts. I'm implementing a compiler for a custom language that translates to C. This language supports nested functions that can reference variables in their parent functions. From what I understand, the best way to implement this is to pass a static link (a pointer to the stack frame of the enclosing function) to each function, from which you can find the variable you want by applying a constant offset.

The problem I'm having is that I can't seem to figure out a way to consistently get the correct offset for a given variable. Here's a quick example:

code:
// This function is nested inside parent and tries to print the value of x inside parent.
void nested(void* staticLink) {
    // Need to figure out the offset at compile time.
    printf("%d\n", *(int*)((long)staticLink + ???));
}

void parent(void* staticLink) {
    int x = 5;
    nested((void*)&staticLink);
}
I've been looking at the addresses of the local variables, but I've been unable to find an identifiable pattern from which to calculate an offset. For example, if I have up to 4 local ints, they seem to be at staticLink + 20, 16, 12, and 8 respectively. But if I have 5 local ints, it's something else entirely.

First of all, am I on the right track with this? I'm not sure if using a pointer to the parent function's first parameter is the "proper" way to implement a static link. Second, how can I determine the offset of a given local variable from the static link at compile time? It'd be a heck of a lot easier if I knew exactly what a given function's stack frame looked like, but I'm not sure how to figure that out.

A nudge in the right direction would be greatly appreciated.

POKEMAN SAM
Jul 8, 2004

Gordon Cole posted:

I've got a homework assignment that's driving me nuts. I'm implementing a compiler for a custom language that translates to C. This language supports nested functions that can reference variables in their parent functions. From what I understand, the best way to implement this is to pass a static link (a pointer to the stack frame of the enclosing function) to each function, from which you can find the variable you want by applying a constant offset.

Just figure out which variables from the parent function are being used in the nested function and pass references/pointers to all of them when calling the nested function.

pseudorandom name
May 6, 2007

Gordon Cole posted:

First of all, am I on the right track with this? I'm not sure if using a pointer to the parent function's first parameter is the "proper" way to implement a static link. Second, how can I determine the offset of a given local variable from the static link at compile time? It'd be a heck of a lot easier if I knew exactly what a given function's stack frame looked like, but I'm not sure how to figure that out.

You can't predict anything about the layout of stack variables. The compiler can (and will) place them in any order it feels like, sometimes combining multiple variables into a single stack slot or never allocating any stack space at all.

Vanadium
Jan 8, 2005

You might wanna use a struct for each stack frame and pass pointers to that around. :shobon:

tractor fanatic
Sep 9, 2005

Pillbug
Out of curiosity, is there a reason that delete and delete[] both exist? Is it an efficiency thing?

Dicky B
Mar 23, 2004

every delete must match the form of its new and doing otherwise results in undefined behaviour.
code:
T* p1 = new T;
delete p1;

T* p2 = new T[10]
delete[] p2;

tractor fanatic
Sep 9, 2005

Pillbug

Dicky B posted:

every delete must match the form of its new and doing otherwise results in undefined behaviour.
code:
T* p1 = new T;
delete p1;

T* p2 = new T[10]
delete[] p2;

yeah, but that's up to the designers of c++. was there a reason it needed to be that way?

Harokey
Jun 12, 2003

Memory is RAM! Oh dear!

tractor fanatic posted:

yeah, but that's up to the designers of c++. was there a reason it needed to be that way?

http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.13

Edit: Although I suppose that link doesn't exactly explain why this decision was made...

Paniolo
Oct 9, 2007

Heads will roll.
code:
#include <iostream>

struct Foo {
   Foo() { std::cout << "Foo::Foo()\n" << std::endl; }
   ~Foo() { std::cout << "Foo::~Foo()\n" << std::endl; }
};

void bar(Foo* ptr)
{
   delete ptr;
}

int main()
{
   Foo* fooArray = new Foo[8];
   bar(fooArray);
   return 0;
}
Consider what this program will output - in particular, how many times Foo's destructor will be called - and the design will make more sense.

tractor fanatic
Sep 9, 2005

Pillbug
But I'm not telling delete[] how many times to call the destructor; it already knows that magically. And like Dicky B said: you can't mix new[] with delete. I can't specifically choose to call the destructor on only the one element, so combining delete and delete[] into a single magical operator doesn't seem like it loses any expressive power.

nielsm
Jun 1, 2009



Paniolo posted:

Consider what this program will output - in particular, how many times Foo's destructor will be called - and the design will make more sense.

But why not just translate new Foo to new Foo[1] in the compiler and only have operator new[], completely scrapping the non-array operator new? And the same with operator delete.
As far as I know, there isn't any practical difference between allocating an array of a single element and just allocating a single instance.

tractor fanatic posted:

But I'm not telling delete[] how many times to call the destructor; it already knows that magically. And like Dicky B said: you can't mix new[] with delete. I can't specifically choose to call the destructor on only the one element, so combining delete and delete[] into a single magical operator doesn't seem like it loses any expressive power.

But what if the library allocates from different pools depending on whether you use the regular or array operator new?

Clobbersaurus
Feb 26, 2004

That Turkey Story posted:

Yes, you can put an instance of any type on the stack in C++ (of course, that type itself might internally allocate memory on the free store if it wants, i.e. an std::map or std::string).

Sorry to be that guy, but even though you haven't given us much information you are pretty much definitely going about whatever it is that you want to do in the wrong way. Your scenario already implies occupying over 2 gigs of memory for storing this data alone. I highly doubt you actually need hundreds of millions of elements all in memory at once. If you're in 32-bit land, as it sounds, you're realistically running out of address space already. Give us some more information and we can probably tell you a different way to approach your problem since explicitly representing hundreds of millions of objects in a map, or really any datastructure in memory, rarely makes sense.

pr0metheus posted:

You could use a hash_map, or an array (which would require changing logic of your program most likely a bit). That should reduce the amount of overall memory needed.

And yea, you should explain your problem to us so we can suggest a better algorithm/data-structure.

Normally I'd be really happy to explain all the details of a project like this, but the main goal of the program is research (I'm looking to publish, based on ~4 months of work), so it seems like a bad idea to post it all on the internet before I'm finished. I could cut out the over-arching explanation, but then I wouldn't be able to explain the justification of the engineering well. I know it probably sounds like I don't know what I'm talking about (I'm kind of rusty in C), but a single large object is unavoidable in this application (and I'd normally scoff at this too). Thanks for your help - I'll come back and tell the whole story if you're still interested when the paper's out.

tractor fanatic
Sep 9, 2005

Pillbug

nielsm posted:

But what if the library allocates from different pools depending on whether you use the regular or array operator new?

Do you mean as a result of an overload, or as a result of the way the system manages the memory?

Adbot
ADBOT LOVES YOU

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

nielsm posted:

But why not just translate new Foo to new Foo[1] in the compiler and only have operator new[], completely scrapping the non-array operator new? And the same with operator delete.
As far as I know, there isn't any practical difference between allocating an array of a single element and just allocating a single instance.

There's no difference in layout, if that's what you mean. There's a difference in semantics at deletion time, where deleting a single object can require a virtual destructor to be called but deleting an array cannot, but again that's directed by the core division. The reason for that division is efficiency and simplicity. While the committee doesn't guarantee that the standard allocation functions will be implemented in terms of malloc and free, it does want to permit that, which would not be possible if the platform ABI required separate memory pools for single-object allocations vs. array allocations. And if the platform ABI doesn't require that, then it's basically forced to allocate extra memory with every single (non-POD) heap allocation.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply