C/C++ Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›639 »

FamDav: Mar 29, 2008

Boz0r posted:

I'm trying to reduce my programs memory usage and so far it's going badly. I have a std::vector of Vector3D(double x,y,z) that I want to convert to a std::vector of pointers to these Vector3Ds as I'm going to copy these a shitload of times in a recursive algorithm.
code:
    std::vector<Vector3D> &input = fromSomewhere();    // n = 2^24. Shoulde be 2^24*(8+8+8) = 384 MB
    // Memory usage 395 MB

    std::vector<std::shared_ptr<Vector3D>> input_pointers = std::vector<std::shared_ptr<Vector3D>>();
    input_pointers.reserve(input.size());
    // 658 MB

    for (int j = 0; j < input.size(); ++j) {
        input_pointers.push_back(std::make_shared<Vector3D>(input.at(j)));
    }
    // 3 GB, what the hell?
What should I be doing here?

depending on the structure of your recursive algorithm and what you require, i would either do Vector3D const *'s like seiken mentioned OR if i'm copying contiguous subranges of the vector use iterator pairs.

how many copies are you expecting to exist at any given time, anyways?

# ? Oct 20, 2014 04:02

Adbot: ADBOT LOVES YOU

# ? Apr 23, 2024 07:32

Boz0r: Sep 7, 2006; The Rocketship in action.

I'm scared of raw pointers and I want to test my algorithm with as high a an input size as possible. It's a divide and conquer algorithm, so I'm copying it lg n times, or something.

# ? Oct 20, 2014 11:42

seiken: Feb 7, 2005; hah ha ha

Boz0r posted:

I'm scared of raw pointers and I want to test my algorithm with as high a an input size as possible. It's a divide and conquer algorithm, so I'm copying it lg n times, or something.

this sounds a bit confused. Are you really worried memory usage, or are you worried about the time impact of doing the copies? It's not really useful to talk about stuff like this until you understand the properties of whatever algorithm you're implementing better than "copying [something] lg n times", which isn't really a thing that makes sense. Also, raw pointers aren't scary

# ? Oct 20, 2014 12:05

Boz0r: Sep 7, 2006; The Rocketship in action.

If I'm making lg n copies of a 6 GB vector, I think I'll run out of memory real fast. Maybe it's time to actually learn proper pointer use, then.

# ? Oct 20, 2014 12:38

nielsm: Jun 1, 2009

Boz0r posted:

If I'm making lg n copies of a 6 GB vector, I think I'll run out of memory real fast. Maybe it's time to actually learn proper pointer use, then.

If your divide-and-conquer algorithm works on non-overlapping slices of the vector, and does not do concurrency either, and you need changes to ordering to be propagated, then work in-place using either indexes or iterators.

Otherwise you can have bare pointers directly to vector elements in other vectors, as long as the original vector does not change size during the lifetime of those vectors-of-pointers.

# ? Oct 20, 2014 12:48

seiken: Feb 7, 2005; hah ha ha

Boz0r posted:

If I'm making lg n copies of a 6 GB vector, I think I'll run out of memory real fast. Maybe it's time to actually learn proper pointer use, then.

You are not going to be making lg n copies of a 6 GB vector.

# ? Oct 20, 2014 15:45

FamDav: Mar 29, 2008

also how are you getting std::vector<Vector3D> &input = fromSomewhere(); to compile w/out const.

# ? Oct 20, 2014 16:53

Vanadium: Jan 8, 2005

Maybe fromSomewhere() returns a reference to a longer-lived vector. :colbert:

# ? Oct 20, 2014 18:19

Precambrian Video Games: Aug 19, 2002

I don't know if this is even a C++ question so much as an Eclipse CDT/Linux shell question, but here goes. I'm running some code that forks to run an external makefile like so:

code:

int fdlog = open(makelog.c_str(), O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
int fderr = open(makeerr.c_str(), O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);

dup2(fdlog,1);
dup2(fderr,2);

execlp("make","make","-f",makefile.c_str(),maketargets.c_str(),(char *)NULL);

It seems to work fine just running the program in the shell and redirects the stdout/err to the specified files. But trying to debug it in Eclipse yields this error:

make: relocation error: make: symbol bsd�signal, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference

bsd_signal is defined:

[]$ objdump -T /usr/lib64/libc.so.6 | grep bsd
00000000000bd7e0 g DF .text 0000000000000005 GLIBC_2.2.5 __bsd_getpgrp
00000000000357a0 w DF .text 000000000000009e GLIBC_2.2.5 bsd_signal

... so, uh, what? Did Eclipse mangle the underscore in bsd_signal to �? I haven't found anything remotely helpful in searching.

I guess if I can't find a solution I'll just give up on debugging it in Eclipse, but it would be nice if it worked.

# ? Oct 20, 2014 19:35

hooah: Feb 6, 2006; WTF?

Why the hell would g++ 4.7.2 seem to be telling me it doesn't know what auto is?? I have a line e.g. auto itr = mLists.begin();, and I get an error "error: 'itr' does not name a type". Well no poo poo, g++, that's because there's auto before it!

# ? Oct 21, 2014 02:46

fritz: Jul 26, 2003

hooah posted:

Why the hell would g++ 4.7.2 seem to be telling me it doesn't know what auto is?? I have a line e.g. auto itr = mLists.begin();, and I get an error "error: 'itr' does not name a type". Well no poo poo, g++, that's because there's auto before it!

Are you explicitly specifying c++11 mode?

# ? Oct 21, 2014 02:52

hooah: Feb 6, 2006; WTF?

fritz posted:

Are you explicitly specifying c++11 mode?

Ahhh crap, I had my CXXFLAGS set, but didn't actually use them anywhere like a dumbass.

Well, maybe. I can't remember how that works with replacement. Does this makefile make sense?

code:

CXX=g++
CXXFLAGS=-std=C++11 -Wall
EXECUTABLE=PROG4

SRC=$(wildcard *.cpp)
OBJ=$(SRC:%.h%.cpp=%.o)

all: $(OBJ)
	$(CXX) -o $(EXECUTABLE) $^

%.o: %.c
	$(CXX) -o $@ -c $<

Edit: Eh, gently caress it, it's too late for this. I'll just use a straightforward makefile instead of trying to be fancy.

hooah fucked around with this message at 03:13 on Oct 21, 2014

# ? Oct 21, 2014 03:05

OddObserver: Apr 3, 2009

You're not actually using CXXFLAGS?

# ? Oct 21, 2014 03:18

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

If you name your files *.cpp then you can use the built-in rule, which picks up CXXFLAGS.

# ? Oct 21, 2014 03:20

OddObserver: Apr 3, 2009

Subjunctive posted:

If you name your files *.cpp then you can use the built-in rule, which picks up CXXFLAGS.

Oh right, that probably is using that. I misread that the all: target entirely.

# ? Oct 21, 2014 03:41

robostac: Sep 23, 2009

Pretty sure it has to be a lower case c in c++11. Though I've just tried it and my version of gcc gives an error with the upper case C so it's still possible your flags aren't being used at all.

# ? Oct 21, 2014 11:53

hackbunny: Jul 22, 2007; I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

Blotto Skorzany posted:

GCC already gives you warnings for arguments that conflict with the types in printf format strings, no idea what the implementation is based on (probably a bunch of builtin crap)

A bunch of builtin crap, specifically a custom attribute to decorate printf-like functions. It's actually pretty flexible and you can apply it to your own printf-like abominations

Vanadium posted:

Is the problem with iostreams vs stdio really just that overloading << is gross and not the more architectural poo poo?

Have you ever tried to write your own stream subclass?

# ? Oct 22, 2014 09:51

BigRedDot: Mar 6, 2008

hackbunny posted:

Have you ever tried to write your own stream subclass?

I remember when I was young, in the 90s, and that seemed like a totally awesome idea.

# ? Oct 22, 2014 21:40

Schmerm: Sep 1, 2000; College Slice

Basic question. Why does this work:

code:

char array[] = {'a', 'b', 'c'};

but not this:

code:

const char* array = {'a', 'b', 'c'};

and why aren't they equivalent, when the following two things ARE equivalent and both compile to the same thing:

code:

char array[] = {'a', 'b', 'c', 0};
const char* array = "abc";

The question being: why does a ""-based literal resolve to a const pointer to a pre-allocated bit of static memory, while the {}-based constant doesn't?

Schmerm fucked around with this message at 19:51 on Oct 28, 2014

# ? Oct 28, 2014 19:39

CRIP EATIN BREAD: Jun 24, 2002; Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T; Soiled Meat

They aren't equivalent (and nor are they compiled to the same thing), although sometimes they appear to be. String literals have static storage.

Remember that char * is NOT the same as char arrays.

code:

#include <stdio.h>

int main(int argc, const char **argv)
{
	char array[] = {'a', 'b', 'c', 0};
	const char *array2 = "abc";
	char array3[] = "abc";

	printf("%p %p %p\n", array, array2, array3);
	printf("%lu %lu %lu\n", sizeof(array), sizeof(array2), sizeof(array3));

	return 0;
}

Look at what sizeof returns.

When you use char *array = "abc" the compiler is storing the location of the string literal in array.
When you use char array[] = {'a', 'b', 'c', 0} the compiler is allocating the four bytes.
When you use char array[] = "abc" the compiler is allocating 4 bytes and is being initialized with the contents of "abc\0".

# ? Oct 28, 2014 22:28

sarehu: Apr 20, 2007; (call/cc call/cc)

Schmerm posted:

The question being: why does a ""-based literal resolve to a const pointer to a pre-allocated bit of static memory, while the {}-based constant doesn't?

Consider the notation

code:

char p[] = "abc";

to be a special case syntax that means

code:

char p[] = {'a', 'b', 'c', 0};

And the normal meaning of "abc" is just syntax for a const pointer to statically allocated char array.

Note that the nested array initializer { {1, 2, 3}, {4, 5, 6} } can never take on the meaning of an array of const pointers to static int arrays. I believe that { "foo", "bar" } as an array initializer can only be used to initialize a const char *arr[], not some sort of char arr[][], but I could be wrong.

# ? Oct 29, 2014 03:09

Deus Rex: Mar 5, 2005

sarehu posted:

Consider the notation
code:
char p[] = "abc";
to be a special case syntax that means
code:
char p[] = {'a', 'b', 'c', 0};
And the normal meaning of "abc" is just syntax for a const pointer to statically allocated char array.

Note that the nested array initializer { {1, 2, 3}, {4, 5, 6} } can never take on the meaning of an array of const pointers to static int arrays. I believe that { "foo", "bar" } as an array initializer can only be used to initialize a const char *arr[], not some sort of char arr[][], but I could be wrong.

I don't think char arr[][] is a valid type, but you can use { "foo", "bar" } to initialize char arr[][3].

# ? Oct 29, 2014 04:41

Boz0r: Sep 7, 2006; The Rocketship in action.

I have a vector containing 100.000 pointers, and I want to free up that memory when I'm done with the vector, but if I just .clear() the vector, the memory usage doesn't decrease in the Windows Task Manager. Shouldn't it do this?

# ? Oct 30, 2014 14:44

nielsm: Jun 1, 2009

The vector remains allocated/reserved, in case you want to fill it again. You might be able to call v.reserve(1) or similar to size the storage down, but I'm not sure if the reserve function is required to decrease the reserved size.

# ? Oct 30, 2014 14:55

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

C++11 has shrink_to_fit.

# ? Oct 30, 2014 14:59

Precambrian Video Games: Aug 19, 2002

Plorkyeran posted:

C++11 has shrink_to_fit.

Note that the documentation also says that the request is nonbinding and that the compiler can optimize it away.

# ? Oct 30, 2014 15:10

The Laplace Demon: Jul 23, 2009; "Oh dear! Oh dear! Heisenberg is a douche!"

Boz0r posted:

I have a vector containing 100.000 pointers, and I want to free up that memory when I'm done with the vector, but if I just .clear() the vector, the memory usage doesn't decrease in the Windows Task Manager. Shouldn't it do this?

eXXon posted:

Note that the documentation also says that the request is nonbinding and that the compiler can optimize it away.

If shrink-to-fit() won't do, there's always "swap-to-fit".

C++ code:

std::vector<Foo*> v; v.reserve(100000);

std::vector<Foo*>{v.begin(), v.end()}.swap(v);

When the temporary goes out of scope, so does that ~780 kB block of memory.

EDIT:

Boz0r posted:

the memory usage doesn't decrease in the Windows Task Manager

I just reread this part. Depending on where it's allocated, there might still be other things around it preventing the program from returning that memory to the OS. You could potentially write your own allocator that calls HeapCreate(), HeapDestroy(), HeapAlloc(), etc so that the vector's in it's own private heap.

On that note, why do you even care about under a megabyte of unnecessary memory usage?

The Laplace Demon fucked around with this message at 17:00 on Oct 30, 2014

# ? Oct 30, 2014 16:30

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Boz0r posted:

I have a vector containing 100.000 pointers, and I want to free up that memory when I'm done with the vector, but if I just .clear() the vector, the memory usage doesn't decrease in the Windows Task Manager. Shouldn't it do this?

Memory usage is surprisingly hard to understand. Even when using C and manually doing free();, there's no guarantee that will actually deallocate the underlying pages, because your malloc implementation might want to reuse them, or there might be memory fragmentation in the way.

# ? Oct 30, 2014 16:31

TheFreshmanWIT: Feb 17, 2012

Boz0r posted:

I have a vector containing 100.000 pointers, and I want to free up that memory when I'm done with the vector, but if I just .clear() the vector, the memory usage doesn't decrease in the Windows Task Manager. Shouldn't it do this?

If you're done with it, why not just let it fall out of scope so that all of its memory is reclaimed?

Additionally (perhaps a silly question!). When you do you vector<Foo*>.clear(), you have already called delete on all of its contents as appropriate, correct? Unless they are smart pointers, clearing the vector will not reclaim that memory.

# ? Oct 30, 2014 16:34

Edison was a dick: Apr 3, 2010; direct current only

Also, returning the memory to the operating system isn't really useful on modern operating systems, since they'll swap out pages that aren't currently being used, and all your memory will be returned to the operating system when your process exits anyway.

This pretty much just leaves long running programs that have fluctuating memory requirements.

# ? Oct 30, 2014 18:27

along the way: Jan 18, 2009

Beginner C questions.

What is the difference between the for and while commands below?

Why do I need to use (a <= b) for the while loop, but (a < b) for the for loop to get the same output? If I use (a < b) for the increment portions of both commands, I get up to 9 for the while command (which makes sense to me logically) and up to 10 for the for command, which I don't understand.

Another small question. Why does the for command begin at the first increment (1 if a = 0 or 2 if a = 1) but the while command begins at the original value of the variable?

code:

#include <stdio.h>

int main()
{
	int a,b;
	for (a=0,b=10; a < b; a++, printf("%d\n",a));
	
	printf("\n");
	
	a=1;
	b=10;
	while (a <= 10)
	{
		printf("%d\n", a);
		a++;
	}
	return 0;
}

along the way fucked around with this message at 18:58 on Oct 30, 2014

# ? Oct 30, 2014 18:50

raminasi: Jan 25, 2005; a last drink with no ice

along the way posted:

Beginner C questions.

What is the difference between the for and while commands below?

Why do I need to use (a <= b) for the while loop, but (a < b) for the for loop to get the same output? If I use (a < b) for the increment portions of both commands, I get up to 9 for the while command (which makes sense to me logically) and up to 10 for the for command, which I don't understand.

Another small question. Why does the for command begin at the first increment (1 if a = 0 or 2 if a = 1) but the while command begins at the original value of the variable?
code:
#include <stdio.h>

int main()
{
	int a,b;
	for (a=0,b=10; a < b; a++, printf("%d\n",a));
	
	printf("\n");
	
	a=1;
	b=10;
	while (a <= 10)
	{
		printf("%d\n", a);
		a++;
	}
	return 0;
}

You are falsely inferring a whole bunch of illusory differences between the two loops because of the fact that in your for loop you increment a before printing it and in the while loop you do the reverse. Swap one of the orders and see if everything falls into place for you.

# ? Oct 30, 2014 19:04

astr0man: Feb 21, 2007; hollyeo deuroga

I think you will get the behavior you were expecting if you make your for loop

C++ code:

for (a = 0, b = 10; a < b; a++)
{
    printf("%d\n", a);
}

This will print the integers 0-9. The way your loop is currently written it will increment a first and then execute the printf, which is why your for loop is currently printing 1-10 instead. The way I have it written above it will execute the printf, and then increment a afterwards.

If you want your for loop to be on one line (which is bad and hard to read imo) it would be

C++ code:

for (a = 0, b = 10; a < b; a++) printf("%d\n", a);

The difference between this and yours is that here the printf is outside the for loop declaration.

# ? Oct 30, 2014 19:05

along the way: Jan 18, 2009

Yup, it was the fact I incremented before printing vs. vice versa. I got it now.

Thanks for the corrections.

# ? Oct 30, 2014 19:17

sarehu: Apr 20, 2007; (call/cc call/cc)

Suspicious Dish posted:

Memory usage is surprisingly hard to understand. Even when using C and manually doing free();, there's no guarantee that will actually deallocate the underlying pages, because your malloc implementation might want to reuse them, or there might be memory fragmentation in the way.

Also because it's a syscall.

# ? Oct 31, 2014 01:57

Bruegels Fuckbooks: Sep 14, 2004; Now, listen - I know the two of you are very different from each other in a lot of ways, but you have to understand that as far as Grandpa's concerned, you're both pieces of shit! Yeah. I can prove it mathematically.

Boz0r posted:

I have a vector containing 100.000 pointers, and I want to free up that memory when I'm done with the vector, but if I just .clear() the vector, the memory usage doesn't decrease in the Windows Task Manager. Shouldn't it do this?

Windows memory management is very complicated and the task manager isn't a great tool for understanding what's going on - use the CRT debug heap functions: http://msdn.microsoft.com/en-us/library/x98tx3cf.aspx. It's not idiot-proof, but you'll generally know if you gently caress up immediately.

Bruegels Fuckbooks fucked around with this message at 03:42 on Oct 31, 2014

# ? Oct 31, 2014 02:04

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

sarehu posted:

Also because it's a syscall.

Wait, the heap isn't part of the CRT?

# ? Oct 31, 2014 03:35

pseudorandom name: May 6, 2007

Actually really freeing the memory requires a VirtualFree() or a munmap() or the equivalent. (And that in turn requires page table modification and TLB shootdowns and whatnot and Is Not Cheap.)

# ? Oct 31, 2014 03:36

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

Releasing memory to the OS is usually a syscall. Heap manages the memory once the OS has provided it.

Releasing memory to the OS can also be a good idea because it helps the OS page the right things out when it's under pressure. (You can madvise too, but it's usually not much better unless you're hurting because of mmap/munmap hysteresis or something.)

# ? Oct 31, 2014 03:38

Adbot: ADBOT LOVES YOU

# ? Apr 23, 2024 07:32

The Gay Bean: Apr 19, 2004

I always try to follow the Ubuntu package system for my dependencies, but then I find that a library is broken in some terrible way and I almost always end up having to compile it from source anyway. Then I have to write convoluted instructions for my coworkers. An example: Ubuntu doesn't provide the opencv_nonfree module for OpenCV because of licensing issues. Also gently caress ffmpeg with a 10 foot pole.

I think I'm just going to start making Cmake compile all of my dependencies with specific flags and git/svn versions.. would anybody hate me if I were their coworker?

# ? Nov 3, 2014 15:28

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›639 »