.NET Megathread 3.5: await GetGoodPosts()

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > .NET Megathread 3.5: await GetGoodPosts()

«‹›298 »

aperfectcirclefan: Nov 21, 2021; by Hand Knit

I think that's a problem with all online tutorials and such. It's either very basic or verya advanced; never the in-between. That's kind of where you're on your own and have to figure out what to do next. So if you had a car app you would want to figure out how to do sorting and filtering. Or make a system that takes people's informations and emails it out.

# ? Mar 9, 2022 13:40

Adbot: ADBOT LOVES YOU

# ? Apr 19, 2024 23:19

distortion park: Apr 25, 2011

fuf posted:

Honestly? I'm trying to learn enough .NET and C# so that I can switch jobs and get a mid-level developer role somewhere.

Which short term means trying to come up with random projects to keep up some motivation.

I think you're right about learning too many things at once. Umbraco is a branch too far at this stage.

I think the wall I hit with my Blazor app wasn't anything technical, but more to do with design patterns and just not really knowing how and where to structure all the business logic.

Blazor components / front end views? Fine with all that
Database operations? Difficult but I know what I'm trying to do
But the layer in between where, you know, the actual app is meant to do its thing? That's where I was really clueless about the best way to structure things. Everything got too mixed together and I would make changes that would cascade and break a bunch of other things. I hit a point where instead of being excited to keep hacking away my heart would sink at the mess of different files.

Because Blazor is new most of the learning resources are like "here is what is new and specific about Blazor components" but not much "here is how to structure an application in general". It also doesn't help that every tutorial course is like "how to add an employee to your list of employees". Or some of them get really adventurous and are like "how to add a car to your list of cars". But none of them really seem to do do anything with the data.

That's a totally reasonable problem to have, plenty of real businesses haven't solved it well in their own codebases! If you're just learning I wouldn't worry about it, just try and get it to do what you want it to do, and if you get annoyed see if you can come up with a better way. It's all very well someone telling you the best way to do something or structure code, but you won't really understand it (or even be able to evaluate if it's a good idea or not) unless you've tried doing it another way.

# ? Mar 9, 2022 14:37

distortion park: Apr 25, 2011

e.g. There's nothing wrong with just serializing your objects to a string field and storing that in the DB for a toy project. You'll learn quickly enough why people don't normally do that, but also when it does make sense!

# ? Mar 9, 2022 14:40

zokie: Feb 13, 2006; Out of many, Sweden

What finally got me to �get� a lot of how to setup the internals of an app was chasing test coverage. Without excessive mocking, or even no mocking which I guess was easier for me because I was doing this with TypeScript.

But the point of this is that getting to 100% test coverage is only ever feasible if you split you code you into modules that make sense. Otherwise you will have to repeat yourself way to much chasing that one branch deep down somewhere. So you realize you don�t want to copy paste some big rear end test with a minor tweak, how do you get out of that? You extract the part containing the branch into its own testable unit.

For me backend/C# is a lot more explorative and I don�t really do TDD so I might not have as many test as when I�m coding the frontend. But the mindset of �how am I going to test this� helps me define boundaries and interfaces.

Also when making the tests, look into AutoFixture for making random test data for you.

# ? Mar 11, 2022 08:18

Small White Dragon: Nov 23, 2007; No relation.

If anyone else is doing .NET on OS X -- is the JetBrains IDE really that much better than Visual Studio, and if so, why?

# ? Mar 12, 2022 09:01

Canine Blues Arooo: Jan 7, 2008; when you think about it...i'm the first girl you ever spent the night with; Grimey Drawer

Small White Dragon posted:

If anyone else is doing .NET on OS X -- is the JetBrains IDE really that much better than Visual Studio, and if so, why?

Rider is decidedly better than VS Code, but VS proper has way, way more features. It probably comes down to what features you want/use. If you use those features, then Rider is gonna fall short. If you do Interface development, then Rider is going to fall way short.

# ? Mar 12, 2022 09:09

mystes: May 31, 2006

If when you say Visual Studio in relation to .net on OS X you mean "Visual Studio for Mac" (i.e. Xamarin Studio) then yeah I would assume that rider will be better, and probably vs code as well.

# ? Mar 12, 2022 20:57

LongSack: Jan 17, 2003

Pretty sure this is an async related question.

I have an extension method:

C# code:

public static void ForEach<T>(this IEnumerable<T> list, Action<T> action)
{
  if (list is null)
  {
    throw new ArgumentNullException(nameof(list));
  }
  if (action is null)
  {
   throw new ArgumentNullException(nameof(action));
  }
  foreach (var item in list)
  {
    action(item);
  }
}

it seems to have issues in asynchronous code. For example, in a data repository, this code works:

C# code:

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  foreach (var offer in offers)
  {
    offer.Bean = await _beanRepository.ReadAsync(offer.BeanId);
    offer.User = await _userRepository.ReadAsync(offer.UserId);
  }
}

but this does not:

C# code:

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  offers.ForEach(async x =>
  {
    x.Bean = await _beanRepository.ReadAsync(x.BeanId);
    x.User = await _userRepository.ReadAsync(x.UserId);
  });
}

Any explanation for why it doesn't work? I am not strong on async stuff. TIA

# ? Mar 14, 2022 15:44

WorkerThread: Feb 15, 2012

Your delegate is async but ForEach isn't, so the rest of your method continues executing without waiting on the result. You could make ForEach async and return a Task, then await that.

# ? Mar 14, 2022 16:44

Red Mike: Jul 11, 2011

The two aren't equivalent. Your last sample is equivalent to this:

C# code:

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  foreach (var item in offers) {
    var task = Task.Run(async () => {
      item.Bean = await _beanRepository.ReadAsync(item.BeanId);
      item.User = await _userRepository.ReadAsync(item.UserId);
    });
    //nothing is awaiting the task, so it'll run in the background
  }
  //nothing is awaiting the started tasks, so if you check here, then nothing has happened
  //what you want here is: await Task.WhenAll(tasks); where tasks is a List<Task>
}

As a general rule, if you're passing an async lambda into an Action<> then you're likely going to mess something up. Anything that can needs to await should be returning Task (because it means you have a way to know when the operation has finished). For example your extension method should be something like:

C# code:

public static IEnumerable<Task> ForEach<T>(this IEnumerable<T> list, Func<T, Task> action)
{
  if (list is null)
  {
    throw new ArgumentNullException(nameof(list));
  }
  if (action is null)
  {
   throw new ArgumentNullException(nameof(action));
  }
  var tasks = new List<Task>();
  foreach (var item in list)
  {
    var task = action(item);
    tasks.Add(task);
  }
  return tasks;
}

...


var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  var tasks = offers.ForEach(async x =>
  {
    x.Bean = await _beanRepository.ReadAsync(x.BeanId);
    x.User = await _userRepository.ReadAsync(x.UserId);
  });
  //right now, the list is in an uncertain state
  await tasks; //caution: this will throw aggregate exceptions
  //now the list is in a certain state, offers contains the end results
}

But more likely what you really want is this:

C# code:

public static async Task ForEach<T>(this IEnumerable<T> list, Func<T, Task> action)
{
  if (list is null)
  {
    throw new ArgumentNullException(nameof(list));
  }
  if (action is null)
  {
   throw new ArgumentNullException(nameof(action));
  }
  foreach (var item in list)
  {
    await action(item); //runs one callback at a time, not all at once! if an exception is thrown it stops halfway!
  }
}

...

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  await offers.ForEach(async x =>
  {
    x.Bean = await _beanRepository.ReadAsync(x.BeanId);
    x.User = await _userRepository.ReadAsync(x.UserId);
  });
  //the list is already in a certain state
}

e: Here's a fiddle that demonstrates better, hopefully.

Red Mike fucked around with this message at 16:55 on Mar 14, 2022

# ? Mar 14, 2022 16:52

Kyte: Nov 19, 2013; Never quacked for this

LongSack posted:

Pretty sure this is an async related question.

I have an extension method:

C# code:

public static void ForEach<T>(this IEnumerable<T> list, Action<T> action)
{
  if (list is null)
  {
    throw new ArgumentNullException(nameof(list));
  }
  if (action is null)
  {
   throw new ArgumentNullException(nameof(action));
  }
  foreach (var item in list)
  {
    action(item);
  }
}

it seems to have issues in asynchronous code. For example, in a data repository, this code works:

C# code:

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  foreach (var offer in offers)
  {
    offer.Bean = await _beanRepository.ReadAsync(offer.BeanId);
    offer.User = await _userRepository.ReadAsync(offer.UserId);
  }
}

but this does not:

C# code:

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())
{
  offers.ForEach(async x =>
  {
    x.Bean = await _beanRepository.ReadAsync(x.BeanId);
    x.User = await _userRepository.ReadAsync(x.UserId);
  });
}

Any explanation for why it doesn't work? I am not strong on async stuff. TIA

Language-based foreach integrates with the async state machine of the method that contains it. It'll loop and stop as needed.
Your ForEach method does not have an async signature, so it'll be run synchronously.
async x => { } produces a Func<T, Task>, so basically your ForEach will go through each element, run your action, get a Task and move on. Since ForEach does not await these tasks, it will not wait for completion before moving onto the next item in the loop. (Hence why the compiler isn't asking you to make the ForEach async)

What you need is:

C# code:

public static Task ForEachAsync<T>(this IEnumerable<T> list, Func<T, Task> asyncAction)
{
  if (list is null)
  {
    throw new ArgumentNullException(nameof(list));
  }
  if (action is null)
  {
   throw new ArgumentNullException(nameof(action));
  }
  foreach (var item in list)
  {
    await action(item);
  }
}

And then you call it with await ForEachAsync(items, async x => {}).

Kyte fucked around with this message at 16:58 on Mar 14, 2022

# ? Mar 14, 2022 16:54

Red Mike: Jul 11, 2011

I guess it's also worth pointing out that the more standard approach to something like this tends to be this pattern:

C# code:

IEnumerable<MyClass> list = await GetData();
IEnumerable<Task<MyClass>> tasks = list
    .Select(async (item) => {
        item.Field1 = await GetFieldData(item.Field1Id);
        item.Field2 = await GetFieldData(item.Field2Id);
        //more commonly you'd construct a new class here, not modify an existing one, but the same applies
        return item;
    });
IEnumerable<MyClass> results = await Task.WhenAll(tasks); //again, this will throw an aggregate exception and not terminate on first exception

I wish at this point it were easier to catch some of these odd edge cases where a Task is started without realising that it's never awaited. Even the IDE won't necessarily highlight some of these, and generally it's just because of the need to e.g. allow a Func<T, Task> to be passed out as Action<T> without any explicit casting or assignment. If you had to explicitly mark it as fire-and-forget (even if it's something like assigning it to the bogus variable), at least it makes you think about the process a bit more.

# ? Mar 14, 2022 20:30

NihilCredo: Jun 6, 2011; iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

Red Mike posted:

I wish at this point it were easier to catch some of these odd edge cases where a Task is started without realising that it's never awaited. Even the IDE won't necessarily highlight some of these, and generally it's just because of the need to e.g. allow a Func<T, Task> to be passed out as Action<T> without any explicit casting or assignment. If you had to explicitly mark it as fire-and-forget (even if it's something like assigning it to the bogus variable), at least it makes you think about the process a bit more.

If you enable the editor setting* to warn on unused local variables, wouldn't that likely cover this as well? There isn't much you can do with a Task object other than awaiting it.

* Apparently it's not available as a compiler warning yet

# ? Mar 14, 2022 20:41

insta: Jan 28, 2009

LongSack posted:

Pretty sure this is an async related question.

I have an extension method:
C# code:
var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters));
if (offers is not null && offers.Any())

I get that this was probably pseudo-code, but know that with some ORMs, .Any() and then a subsequent foreach will re-query the database. The safer way to do that is track state if you've processed any items and early-return, then do whatever the 'Any == false' case is afterwards.

# ? Mar 14, 2022 20:59

Red Mike: Jul 11, 2011

I mean cases like in this one, where you pass in an async lambda (that implicitly returns Task) to something that expects an Action<T>, therefore nothing flags up that you might start a fire-and-forget task when the action is invoked.

What I'm suggesting would be:

C# code:

public void MyMethod(Action<int> callback) {
  callback(1);
  callback(2);
  callback(3);
}

...

//this silently is allowed but starts fire-and-forget tasks
MyMethod(async (i) => {
  await Task.Delay(1000);
  Console.WriteLine("callback " + i);
});
//output takes ~1s and is all at once:
//    callback 1
//    callback 2
//    callback 3

//it would be good if the above raised an error/warning instead to require explicit casting
MyMethod((Action<int>)(async (i) => {
  await Task.Delay(1000);
  Console.WriteLine("callback " + i);
}));

//or even if you couldn't cast it at all (but if it applied to any Func vs Action, that's extreme) and it explicitly required a wrapper
MyMethod((i) => {
  Task.Run(async () => {
    await Task.Delay(1000);
    Console.WriteLine("callback " + i);
  });
});

I get that it's not likely to be possible anyway because not requiring casting between Func and Action is ideal for a lot of cases; it's just that fitting the Task/async stuff into it after-the-fact has made it a bit confusing for these edge cases.

# ? Mar 14, 2022 21:01

Kyte: Nov 19, 2013; Never quacked for this

I think a good rule of thumb is that if you're passing an async lamba in first place you should be checking if the method can actually do anything with it. The async keyword kinda stands out.

# ? Mar 14, 2022 22:05

LongSack: Jan 17, 2003

Thanks for all the responses. It makes a lot more sense now.

insta posted:

I get that this was probably pseudo-code, but know that with some ORMs, .Any() and then a subsequent foreach will re-query the database. The safer way to do that is track state if you've processed any items and early-return, then do whatever the 'Any == false' case is afterwards.

I�m using Dapper which just executes SQL that I pass in, so I�m pretty sure this isn�t an issue. (Also not pseudo-code)

# ? Mar 14, 2022 23:04

insta: Jan 28, 2009

LongSack posted:

Thanks for all the responses. It makes a lot more sense now.

I�m using Dapper which just executes SQL that I pass in, so I�m pretty sure this isn�t an issue. (Also not pseudo-code)

Dapper specifically executes an ExecuteReader each time the enumeration starts. You're querying twice

# ? Mar 14, 2022 23:14

Nth Doctor: Sep 7, 2010; Darkrai used Dream Eater!
It's super effective!

insta posted:

I get that this was probably pseudo-code, but know that with some ORMs, .Any() and then a subsequent foreach will re-query the database. The safer way to do that is track state if you've processed any items and early-return, then do whatever the 'Any == false' case is afterwards.

insta posted:

Dapper specifically executes an ExecuteReader each time the enumeration starts. You're querying twice

That would then make it a good idea to materialize your enumerable as a list:

code:

var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters)).ToList();

That way you uphold the contract of only enumerating your enumerable once, but can still repeatedly use your collection of results.

# ? Mar 15, 2022 16:12

insta: Jan 28, 2009

Nth Doctor posted:

That would then make it a good idea to materialize your enumerable as a list:
code:
var offers = await conn.QueryAsync<OfferEntity>(sql, BuildParameters(parameters)).ToList();
That way you uphold the contract of only enumerating your enumerable once, but can still repeatedly use your collection of results.

The downside is you have to hold the entirety of the database result set in memory, which is potentially a lot of overhead if you're just stream-processing the results. Better IMO to emulate your .Any() behavior in another manner, wouldn't it be? A construct similar to:

C# code:


var query = db.QueryAsync<MyEntity>(sql);
var anyResults = false;

foreach(var item in query) {
   anyResults = true;
   DoWhatever(item);
}

if (anyResults)
   return;

throw new NoItemsProcessedException(sql);

# ? Mar 15, 2022 17:45

LongSack: Jan 17, 2003

insta posted:

Dapper specifically executes an ExecuteReader each time the enumeration starts. You're querying twice

Was not aware. I will adjust my behavior accordingly. I just started using Dapper after long-running frustrations with EF Core.

Is there a way using MS SSMS to see all incoming queries? I think I might want to check some other parts of the code to make sure that what I think is hitting the database is actually hitting the database.

# ? Mar 15, 2022 18:48

insta: Jan 28, 2009

LongSack posted:

Was not aware. I will adjust my behavior accordingly. I just started using Dapper after long-running frustrations with EF Core.

Is there a way using MS SSMS to see all incoming queries? I think I might want to check some other parts of the code to make sure that what I think is hitting the database is actually hitting the database.

Dapper exposes a configuration object that you can implement an OnCompleted callback onto. I don't know exactly where or how, but that will fire every time a query executes. You can possibly inspect the SQL sent as well as see how many times it executed.

# ? Mar 15, 2022 18:55

Jen heir rick: Aug 4, 2004; when a woman says something's not funny, you better not laugh your ass off

LongSack posted:

Was not aware. I will adjust my behavior accordingly. I just started using Dapper after long-running frustrations with EF Core.

Is there a way using MS SSMS to see all incoming queries? I think I might want to check some other parts of the code to make sure that what I think is hitting the database is actually hitting the database.

You can use Sql Server Profiler to see requests happen in realtime. But also, Dapper does not work this way. The way you can tell is if the method returns an IQueryable. QueryAsync returns an IEnumerable. Which means that all of the results are returned in one go and held in memory.

# ? Mar 15, 2022 19:50

insta: Jan 28, 2009

Jen heir rick posted:

You can use Sql Server Profiler to see requests happen in realtime. But also, Dapper does not work this way. The way you can tell is if the method returns an IQueryable. QueryAsync returns an IEnumerable. Which means that all of the results are returned in one go and held in memory.

That is absolutely not how IQueryable or IEnumerable works.

# ? Mar 15, 2022 20:16

Jen heir rick: Aug 4, 2004; when a woman says something's not funny, you better not laugh your ass off

insta posted:

That is absolutely not how IQueryable or IEnumerable works.

It's not? How so?

# ? Mar 15, 2022 20:22

insta: Jan 28, 2009

IQueryable is an extension on IEnumerable to allow for better building of queries, but ultimately they work the same. IEnumerable exposes 2 items: MoveNext() and Current. If MoveNext returns true, Current has a valid value. You call MoveNext until it returns false, and consume Current. This is what foreach loops do behind the scenes, and nothing about this interface makes any mention of how the data is stored. Lists, Arrays, HashSets, etc can easily implement this with their in-memory collections, but Dapper does not.

If we look at SqlMapper.Async.cs, where Dapper actually does its work: https://github.com/DapperLib/Dapper/blob/main/Dapper/SqlMapper.Async.cs#L406

Line 449 calls into the code that actually does the query (ExecuteReaderSync), and gives us the enumerable production:

C# code:

        private static IEnumerable<T> ExecuteReaderSync<T>(IDataReader reader, Func<IDataReader, object> func, object parameters)
        {
            using (reader)
            {
                while (reader.Read())
                {
                    yield return (T)func(reader);
                }
                while (reader.NextResult()) { /* ignore subsequent result sets */ }
                (parameters as IParameterCallbacks)?.OnCompleted();
            }
        }

The 'yield return' keyword satisfies the MoveNext / Current of the consumer on-demand, and this code disposes the database connection after the first iteration, regardless of how far you get through it. The LINQ method .Any() will basically return the result of "MoveNext()" (if you can MoveNext, you have at least 1 item, which satisfies an empty Any()), and to do this, we are executing the query and returning the first result, then disposing the database. Since the connection is now disposed, foreaching through it opens a new connection (although it may be from a pool) and re-executes the query.

It looks like Dapper does support a Buffered=true on the CommandDefinition, which causes it to put it into a List<T> for you, but this still holds the entire result set in memory regardless. OP's code didn't care about that, it just wanted a condition of "did we process rows or not", which could be done with a boolean flag inside of the foreach consuming the data.

# ? Mar 15, 2022 20:39

Jen heir rick: Aug 4, 2004; when a woman says something's not funny, you better not laugh your ass off

I don't see how all that contradicts what I said.

# ? Mar 15, 2022 20:46

insta: Jan 28, 2009

Jen heir rick posted:

I don't see how all that contradicts what I said.

Because you said it holds it in memory, and unless you specifically tell Dapper to do that, Dapper will instead return the items one-by-one from the database reader instead of holding them in memory.

# ? Mar 15, 2022 20:48

bobua: Mar 23, 2003; I'd trade it all for just a little more.

I always just think of an iqueryable return type as part of a query. You can execute it, or further define it, but it isn't a hit on the database until executed. A returned ienumerable is guaranteed to be the result of a query, an actual call to the database.

That's what I assumed jen meant by 'in memory.'

# ? Mar 15, 2022 20:58

insta: Jan 28, 2009

bobua posted:

I always just think of an iqueryable return type as part of a query. You can execute it, or further define it, but it isn't a hit on the database until executed. A returned ienumerable is guaranteed to be the result of a query, an actual call to the database.

That's what I assumed jen meant by 'in memory.'

Just like IQueryable, an IEnumerable isn't executed until you iterate it. If you cast an IQueryable to an IEnumerable, any further LINQ methods are applied after the query happens and can't be optimized by the database provider. A materialized collection type (List, Collection, etc) is guaranteed to have been executed -- but just IEnumerable isn't.

# ? Mar 15, 2022 21:09

Jen heir rick: Aug 4, 2004; when a woman says something's not funny, you better not laugh your ass off

insta posted:

Because you said it holds it in memory, and unless you specifically tell Dapper to do that, Dapper will instead return the items one-by-one from the database reader instead of holding them in memory.

I see what you mean. What I meant was that it holds onto the values once you iterate over the IEnumerable and does not retrieve them from the database again. I just tested this by calling QueryAsync and then iterating over the result set twice. I had Sql Server Profiler running and it only recorded a single query to the database. I don't think that code you posted is correct. I dissasembled the code using Linqpad (Linqpad allows you to jump right to the definition of a method) and it looks like it is executing a reader and holding on to that. Maybe I'm using a different version.

# ? Mar 15, 2022 21:11

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

I used Dapper on an ETL thing the other month and it was pretty clearly hydrating the whole result set. Doubled the number of items to process in one pass and memory use before the transform step was ever called doubled. 4x -> 4x memory use, etc. Didn't bother digging into why, but I thought I sure wasn't calling ToList, Count or Any anywhere.

# ? Mar 15, 2022 21:18

insta: Jan 28, 2009

Jen heir rick posted:

I see what you mean. What I meant was that it holds onto the values once you iterate over the IEnumerable and does not retrieve them from the database again. I just tested this by calling QueryAsync and then iterating over the result set twice. I had Sql Server Profiler running and it only recorded a single query to the database. I don't think that code you posted is correct. I dissasembled the code using Linqpad (Linqpad allows you to jump right to the definition of a method) and it looks like it is executing a reader and holding on to that. Maybe I'm using a different version.

Check the underlying return type of the object, if you can. Dapper has a Buffered option that will fill a List and give you the List, which can be iterated over repeatedly without a second query, but it's still IMO dangerous in that flipping a boolean somewhere can cause multiple queries to start executing again.

# ? Mar 15, 2022 21:18

Jen heir rick: Aug 4, 2004; when a woman says something's not funny, you better not laugh your ass off

insta posted:

Check the underlying return type of the object, if you can. Dapper has a Buffered option that will fill a List and give you the List, which can be iterated over repeatedly without a second query, but it's still IMO dangerous in that flipping a boolean somewhere can cause multiple queries to start executing again.

Just checked this. It is returning a List object whether or not you specify the Buffered option or not. If you do not pass in Buffered=true then upon the second iteration you will get the error: Invalid attempt to call Read when reader was closed. Additionally the query is executed as soon as I call QueryAsync. Not upon iteration. When you iterate it's just calling the data reader methods. I also made sure I'm using the latest version of Dapper.

# ? Mar 15, 2022 21:41

NihilCredo: Jun 6, 2011; iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

Does anybody know of either a logging framework or (ideally) some Serilog plugin that will let me increase log level in a particular execution path alone?

Serilog currently allows me to add extra properties in a given execution path via LogContext.PushProperty(), but not to change the log level.

Basically I would want to do this:

C# code:

		

void SharedMethod() {
    Log.Verbose("Calling shared method");
}

void DoThing1() {
    Log.Debug("Doing thing #1");
    SharedMethod();
}

void DoThing2() {
    Log.Debug("Doing thing #2");
    SharedMethod();
}

async void DoBoth() {
      
    var task1 = async () => {
        using (LogContext.PushProperty("A", 1))
        {
             DoThing1();
        }
   };

    var task2 = async () => {
        using (LogContext.PushProperty("A", 2))
        using (SomeWayToSetTheLogLevel(LogLevel.Verbose))
        {
             DoThing2();
        }
   };
    
   await Task.WhenAll(task1, task2);
         
}

/* should output:

{ A: 1, Level : Information, Message : "Doing thing #1" } 
{ A: 2, Level : Information, Message : "Doing thing #2" } 
{ A: 2, Level : Verbose, Message : "Calling shared method" } 

*/

# ? Mar 16, 2022 13:05

Bruegels Fuckbooks: Sep 14, 2004; Now, listen - I know the two of you are very different from each other in a lot of ways, but you have to understand that as far as Grandpa's concerned, you're both pieces of shit! Yeah. I can prove it mathematically.

NihilCredo posted:

C# code:

		

void SharedMethod() {
    Log.Verbose("Calling shared method");
}

void DoThing1() {
    Log.Debug("Doing thing #1");
    SharedMethod();
}

void DoThing2() {
    Log.Debug("Doing thing #2");
    SharedMethod();
}

async void DoBoth() {
      
    var task1 = async () => {
        using (LogContext.PushProperty("A", 1))
        {
             DoThing1();
        }
   };

    var task2 = async () => {
        using (LogContext.PushProperty("A", 2))
        using (SomeWayToSetTheLogLevel(LogLevel.Verbose))
        {
             DoThing2();
        }
   };
    
   await Task.WhenAll(task1, task2);
         
}

/* should output:

{ A: 1, Level : Information, Message : "Doing thing #1" } 
{ A: 2, Level : Information, Message : "Doing thing #2" } 
{ A: 2, Level : Verbose, Message : "Calling shared method" } 

*/

Serilog has LoggingLevelSwitch, which would let you change log level at runtime. Could do something like: https://nblumhardt.com/2014/10/dynamically-changing-the-serilog-level/. The thing though is that might work nicely if we assume everything is synchronous, but not sure how nicely that works if logging has to happen on multiple threads... Increasing the log level might be ok with that if you're willing to live with other threads occasionally logging something with an increased level.

# ? Mar 16, 2022 21:22

NihilCredo: Jun 6, 2011; iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

Bruegels Fuckbooks posted:

Serilog has LoggingLevelSwitch, which would let you change log level at runtime. Could do something like: https://nblumhardt.com/2014/10/dynamically-changing-the-serilog-level/. The thing though is that might work nicely if we assume everything is synchronous, but not sure how nicely that works if logging has to happen on multiple threads... Increasing the log level might be ok with that if you're willing to live with other threads occasionally logging something with an increased level.

Yeah, I have the runtime switch, but multithreading Is the problem. This is our cronjob service, and it's running dozens of different jobs, so if we want to "drill down" into a particular one by increasing the switch to Debug/Verbose then all of them start logging at Debug/Verbose and it produces an absolute shitload of log data.

It's fine to do so for a bit, but if I could set the log level per-job we could leave a new / buggy / complicated job on Verbose level for a week or so and look for anomalies without running up our ingestion costs with terabytes of worthless crap from the other stable jobs.

The most direct solution is to get rid of the global ILogger instance and instead spawn a dedicated instance (which comes with its own level switch) for each job. This requires injecting it as a parameter into literally every single object or function that might want to emit a log, including common libraries. Great for purity but one heck of a refactor task.

(Comedy option: since almost all our jobs already operate in the AsyncSeq monad (IAsyncEnumerable, basically), I'm almost tempted to write a custom one that combines it with the Reader monad. Would be a giant type tetris mess though.)

Hmm, I might have a legit use case for the service locator pattern? If I take a look at how Serilog implements the execution-context-scoped PushProperty, I might be able to use the same technique to push an ILogger instance into the execution context, and then alias the Log property accessor to locate that instance if one exists (falling back to the global ILogger otherwise).

# ? Mar 16, 2022 23:10

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

Can you wire up a second log sink that accepts logs at a more verbose level from the paths of interest?

# ? Mar 17, 2022 01:50

zokie: Feb 13, 2006; Out of many, Sweden

Get rid of the global logger instance, in aspnetcore you can set log levels on namespaces look into doing that but I guess a single global logger instance is going to be a problem.

# ? Mar 17, 2022 07:29

Adbot: ADBOT LOVES YOU

# ? Apr 19, 2024 23:19

Kyte: Nov 19, 2013; Never quacked for this

NihilCredo posted:

Yeah, I have the runtime switch, but multithreading Is the problem. This is our cronjob service, and it's running dozens of different jobs, so if we want to "drill down" into a particular one by increasing the switch to Debug/Verbose then all of them start logging at Debug/Verbose and it produces an absolute shitload of log data.

It's fine to do so for a bit, but if I could set the log level per-job we could leave a new / buggy / complicated job on Verbose level for a week or so and look for anomalies without running up our ingestion costs with terabytes of worthless crap from the other stable jobs.

The most direct solution is to get rid of the global ILogger instance and instead spawn a dedicated instance (which comes with its own level switch) for each job. This requires injecting it as a parameter into literally every single object or function that might want to emit a log, including common libraries. Great for purity but one heck of a refactor task.

(Comedy option: since almost all our jobs already operate in the AsyncSeq monad (IAsyncEnumerable, basically), I'm almost tempted to write a custom one that combines it with the Reader monad. Would be a giant type tetris mess though.)

Hmm, I might have a legit use case for the service locator pattern? If I take a look at how Serilog implements the execution-context-scoped PushProperty, I might be able to use the same technique to push an ILogger instance into the execution context, and then alias the Log property accessor to locate that instance if one exists (falling back to the global ILogger otherwise).

This is only a vague idea but what about replacing the global with something that hangs off the current synchronization context, and then set up a new context with a new logger for the code path you want to log differently? Async/await should flow the context across invocations (unless you've got everything with ConfigureAwait(false) in which case rip I guess).

Or maybe an async local or thread local depending on whether it's async or not.

These are all hacky, but if you want to avoid a major restructuring it'd at least give you global-like behavior without making it fully global.

# ? Mar 17, 2022 07:50

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > .NET Megathread 3.5: await GetGoodPosts()

«‹›298 »