Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
mystes
May 31, 2006

nielsm posted:

To display a map with your own overlay, you can look into OpenStreetMap.
You can't just use openstreetmap directly (you aren't allowed to use its tile servers for your own apps). You either have to do some very complicated stuff to convert the map for the whole world to tiles and host them yourself or use a third party tile provider service.

Edit: Basically unless you want to use google maps or mapbox, you should probably start by using leaflet, openlayers, or MapLibre with a tile provider (I think a bunch of them have free tiers), and then you can always switch between them or try to self host the tiles later. It's getting easier to self host the tiles so even if you don't want to that now it may be completely trivial soon and if you're using one of those map libraries it should be easy to switch whenever you want to.

mystes fucked around with this message at 21:29 on Apr 5, 2024

Adbot
ADBOT LOVES YOU

Funking Giblet
Jun 28, 2004

Jiglightful!

CitizenKeen posted:


I wrote the app using Razor Pages as an HTMX experiment; had I known what I was doing I probably would have used Vue with a more traditional API model. Not writing any Javascript was a fun idea when it was just for fun.


I wouldn't rush to swap to Vue, you can do a lot with simple HTML + HTMX with a little JavaScript. I've built a large ecom platform and we are removing all of our React and moving to more static file approach with HTMX. The performance increase in rendering is probably 100x.

Mr Shiny Pants
Nov 12, 2012

Funking Giblet posted:

I wouldn't rush to swap to Vue, you can do a lot with simple HTML + HTMX with a little JavaScript. I've built a large ecom platform and we are removing all of our React and moving to more static file approach with HTMX. The performance increase in rendering is probably 100x.

I did not keep up with the Javascript/Web stuff that much, but from what I've seen from HTMX it looks pretty elegant.

mystes
May 31, 2006

If you already want to render everything on the server but then do progressive enhancement primarily through dynamically loading html fragments rather than reloading the whole page, like what used to be common, htmx is a very good option in 2024 for doing that without having to manually write any javascript, but imo it's not necessarily as novel or exciting as some people are making it out to be.

I don't think it's helpful to even think of it as "vue" vs "htmx" as competing frameworks or something. The first question is whether you want to write something as an SPA / render stuff on the client (but maybe add back server rending using something like next.js for SEO or performance) or use traditional server rendering, and if you choose the latter, htmx may be useful.

I do think that some of the htmx hype is because people who don't know what the web was like 20 years ago and are only used to stuff like react or vue are looking at traditional server rendering for the first time and only seeing the ways it seems to be simpler than having a separate frontend making api calls, but maybe not seeing the other ways that approach can actually be more complicated (e.g. having to have a bunch of view state temporarily stored on the server for each session when you're doing anything remotely complicated). But that's not to say that it's a bad approach either; it worked for a really long time and it can still work perfectly fine in 2024.

mystes fucked around with this message at 19:20 on Apr 14, 2024

CitizenKeen
Nov 13, 2003

easygoing pedant
Responses:

Sorry, still looping this thread into my "check up on". My bad.

Yeah, the page is mostly just data retrieval, with one huge form. That form might be better as an SPA, but I don't think it's worth turning the whole page into an SPA. I get a lot of positive feedback on how thin the site is, so I think I made the right call.

I am running into a problem though, and I have no idea where to begin solving it. This is mostly devops, stuff, though - if there's a better thread, I apologize. I think I'm running into some scaling problems. I've gone from 1,000 users in February to 24,000 now, so it's been a bit wild.

I'm seeing these spikes in my processor and my users are reporting intermittent 502s...

https://imgur.com/a/gd3Z3W6

I have no idea how to debug that or where to begin.

I'm running a .Net 7 Razor pages app on a Digital Ocean server with Postgres. That's about it, pretty bog standard. I've made a few small changes to the nginx config based on googling to increase some cache buffer sizes, but otherwise it's mostly out of the box. No cron jobs, no scheduled recurring processes. Just Entity Framework queries and a few views for intensive reads.

(I'm also running a Discord bot using DSharpPlus, but that's mostly a wrapper around two commands which pull some basic static info from my site.)

I don't know if Postgres was the right choice - it was free and its syntax matches SQL Server which I'm used to at work - because my understanding is that it's optimized for writes instead of reads and my site is pretty read heavy.

I'm kind of lost on how to start tracking down why my server's CPU just starts running away. (It's an 8 gig / 4 Intel basic CPU box on Digital Ocean). I can afford to double those resources if I'm just running into resource limits, but I'm concerned it'll be like urban traffic and just swell to fill whatever new stuff I provision.

Any thoughts on where to start investigating this? Please and thank you.

redleader
Aug 18, 2005

Engage according to operational parameters
can you set up your hosting thing to capture a process dump if cpu spikes above 90% or something like that? we've done that with our azure thing in the past, but no idea what sort of tooling digital ocean offers in that regard

can you correlate request count or duration with the spikes?

could there be users with an unusually large number of somethings? example: at a previous job we had strange intermittent perf problems that ultimately arose from a bot who had added thousands of items to its cart (normal number of items would have been less than 20 or something like that)

CitizenKeen
Nov 13, 2003

easygoing pedant

redleader posted:

can you set up your hosting thing to capture a process dump if cpu spikes above 90% or something like that? we've done that with our azure thing in the past, but no idea what sort of tooling digital ocean offers in that regard

can you correlate request count or duration with the spikes?

could there be users with an unusually large number of somethings? example: at a previous job we had strange intermittent perf problems that ultimately arose from a bot who had added thousands of items to its cart (normal number of items would have been less than 20 or something like that)

So the site is hosted on a DigitalOcean VM, so I have pretty hefty ability to add what I want to it, but it's just a hosted Linux box so I don't have true god mode.

The problem is, I don't know what kind of tooling or tracing or anything I should be pursuing. I started this project to learn some industry practices that my management team doesn't value enough, but this scaled faster than I expected.

All I've got in place is Serilog.

I have a search syntax DSL where users can write very simple queries against the data; I'm wondering if I left some infinite regression in there or something. I'm going to have Serilog write out all queries to a separate log file so I can look over what kind of queries people are writing.

Right now, the things I need to learn are:
  1. How do I evaluate my code in production?
  2. When do I know that my code is good enough and the bottleneck is needing to buy more hardware, versus needing to figure out bad loops in code.

I also need to set up a staging environment; that would probably alleviate some of my stress.

nielsm
Jun 1, 2009



You evaluate your code in production by instrumenting it with logging that captures data that can help answer your questions, without interrupting the normal request handling. What data your need to capture depends on the application.

For example, capture the total request times (request received until response ready to be sent) together with some category of the requests, so you can measure whether you have outliers where some requests take unusually long time compared to other requests of the same type, or if the request processing time increases when the service is busy.

Depending on the nature of the data you process, you can also try capturing request series and replay them in a testing environment, where you can more heavily instrument things.


Figuring out whether it's worth optimizing your program more, or throw more hardware at it, is not a science. It's just as much a business tradeoff: What's more expensive, your development time, or your increased hosting bill?
It's probably always a good idea to try to identity the cause of slowness, to decide if it's something you can reasonably improve on or not, and whether it's something that could prevent scaling to faster/wider hardware.

CitizenKeen
Nov 13, 2003

easygoing pedant

nielsm posted:

You evaluate your code in production by instrumenting it with logging that captures data that can help answer your questions, without interrupting the normal request handling. What data your need to capture depends on the application.

Is that something I can do with Serilog, or is that a separate kind of logging?

nielsm
Jun 1, 2009



CitizenKeen posted:

Is that something I can do with Serilog, or is that a separate kind of logging?

Probably? The main thing is to make sure you can retrieve the logged data so you can graph or analyze it in a useful way.
You also need to make sure the instrumentation logging doesn't drown out other operational or error logging, perhaps by logging them to separate streams/files/tables.

If you need to do instrumentation that captures lots of data, it might be better to make a database table/whatever specifically for that data, so you have it structured, efficiently packed, and indexed.

Rawrbomb
Mar 11, 2011

rawrrrrr
If you think the issue is with the user written queries to the database, you could enable some query logging for queries that are taking over a set amount of time. I'm pretty sure PGSQL has some settings to enable and capture this. Might be another option to help you figure out what's going wrong.

CitizenKeen
Nov 13, 2003

easygoing pedant

Rawrbomb posted:

If you think the issue is with the user written queries to the database, you could enable some query logging for queries that are taking over a set amount of time. I'm pretty sure PGSQL has some settings to enable and capture this. Might be another option to help you figure out what's going wrong.

Oh yeah, that's probably a good idea; I'll investigate what PGSQL has. Then I'll have to parse backwards from the SQL that Postgres is running to the EntityFramework it's generating - that might be a mess, but it's my mess.

ChocolatePancake
Feb 25, 2007
You can also configure entity framework to log all of its sql queries. https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/

redleader
Aug 18, 2005

Engage according to operational parameters

CitizenKeen posted:

Oh yeah, that's probably a good idea; I'll investigate what PGSQL has. Then I'll have to parse backwards from the SQL that Postgres is running to the EntityFramework it's generating - that might be a mess, but it's my mess.

you can use IQueryable<T>.TagWith to make it way easier to track sql back to the EF that it came from

Red Mike
Jul 11, 2011

redleader posted:

you can use IQueryable<T>.TagWith to make it way easier to track sql back to the EF that it came from

It sounds like it's not applicable in this case, but just a heads up that this uses SQL comments to do the tagging, which means some managed SQL services won't show it because the comments get stripped out along the way (e.g. Azure SQL's query explorer/monitoring). You can replicate this manually via adding bogus checks (like WHERE 5=5) to your statements, but it's a pain to do with EF even for individual queries.

CapnAndy
Feb 27, 2004

Some teeth long for ripping, gleaming wet from black dog gums. So you keep your eyes closed at the end. You don't want to see such a mouth up close. before the bite, before its oblivion in the goring of your soft parts, the speckled lips will curl back in a whinny of excitement. You just know it.
I've got a mobile app to rewrite at work and a really generous deadline, so I figured I'd take time and play with all the new toys, see what they're like. So I made a MAUI/Blazor hybrid app, and... this is a website. This is a literal, css-using, wwwroot-folder-having, website. They've somehow tricked it into running as a desktop or mobile app, but the code itself is pure website.

And I get what Microsoft is going for, right? If they can unify every design case into "just write it in this one language, we'll compile it appropriately for you", everyone's life gets appreciably easier. But it's been a loving decade since I did web development in any major capacity and it's a bit annoying that we've taken in all of web design's weaknesses too and jettisoned strengths of coding for apps, so now the codebehind just plain can't see anything you put on the razor pages, hope you didn't need sender in any of your event args.

Is it known to what degree this is gonna be the way things go in the future? Like, if Microsoft intends to continue the pressure to turn everything into a Blazor app, okay, gently caress it, I'll start learning web design. But if it's not, I... kinda don't want to.

ChocolatePancake
Feb 25, 2007
It does seem to be the way the world is going, with electron apps, react native, and all that good stuff. No one wants to learn how to write native apps for both iOS and Android and maintain the two code bases. Unless you have a career entirely in stuff that doesn't use the web, you should probably learn some web design.

That said though, I doubt that Maui/Blazor is 'the' future. Blazor maybe, at least for non-tech companies it is very popular, but Maui really isn't good yet, and they don't have many people working on it. Doesn't seem like a big priority for them.

CapnAndy
Feb 27, 2004

Some teeth long for ripping, gleaming wet from black dog gums. So you keep your eyes closed at the end. You don't want to see such a mouth up close. before the bite, before its oblivion in the goring of your soft parts, the speckled lips will curl back in a whinny of excitement. You just know it.

ChocolatePancake posted:

That said though, I doubt that Maui/Blazor is 'the' future. Blazor maybe, at least for non-tech companies it is very popular, but Maui really isn't good yet, and they don't have many people working on it. Doesn't seem like a big priority for them.
I mean, Xamarin is straight up EoL and out of support now; that's the entire reason I have a mobile app to rewrite, the old one was in Xamarin and it turns out it doesn't know how to handle local time zones in Android 14+. And MAUI seems pretty good. Is it 90% still Xamarin? Yeah, but the changes are all good, everything they've done has reduced the amount of platform-specific code you still need to write and platform switching is way cleaner as a result.

mystes
May 31, 2006

CapnAndy posted:

I've got a mobile app to rewrite at work and a really generous deadline, so I figured I'd take time and play with all the new toys, see what they're like. So I made a MAUI/Blazor hybrid app, and... this is a website. This is a literal, css-using, wwwroot-folder-having, website. They've somehow tricked it into running as a desktop or mobile app, but the code itself is pure website.

And I get what Microsoft is going for, right? If they can unify every design case into "just write it in this one language, we'll compile it appropriately for you", everyone's life gets appreciably easier. But it's been a loving decade since I did web development in any major capacity and it's a bit annoying that we've taken in all of web design's weaknesses too and jettisoned strengths of coding for apps, so now the codebehind just plain can't see anything you put on the razor pages, hope you didn't need sender in any of your event args.

Is it known to what degree this is gonna be the way things go in the future? Like, if Microsoft intends to continue the pressure to turn everything into a Blazor app, okay, gently caress it, I'll start learning web design. But if it's not, I... kinda don't want to.
You can also just use MAUI. Hybrid apps are specifically for when you want to use MAUI to present a webview control for your blazor app

That's not to say I think MAUI is good or anything but the whole idea of hybrid apps is to use MAUI to do something like Cordova for blazor apps but within the MAUI / .net ecosystem and within that context the idea probably more or less makes sense.

mystes fucked around with this message at 19:47 on Apr 25, 2024

Rawrbomb
Mar 11, 2011

rawrrrrr

CapnAndy posted:

I've got a mobile app to rewrite at work and a really generous deadline, so I figured I'd take time and play with all the new toys, see what they're like. So I made a MAUI/Blazor hybrid app, and... this is a website. This is a literal, css-using, wwwroot-folder-having, website. They've somehow tricked it into running as a desktop or mobile app, but the code itself is pure website.

And I get what Microsoft is going for, right? If they can unify every design case into "just write it in this one language, we'll compile it appropriately for you", everyone's life gets appreciably easier. But it's been a loving decade since I did web development in any major capacity and it's a bit annoying that we've taken in all of web design's weaknesses too and jettisoned strengths of coding for apps, so now the codebehind just plain can't see anything you put on the razor pages, hope you didn't need sender in any of your event args.

Is it known to what degree this is gonna be the way things go in the future? Like, if Microsoft intends to continue the pressure to turn everything into a Blazor app, okay, gently caress it, I'll start learning web design. But if it's not, I... kinda don't want to.

I'd love to know where y'all land, I'm in a similar situation at work. EOL Xamarin App, and looking to move to something else.

CapnAndy
Feb 27, 2004

Some teeth long for ripping, gleaming wet from black dog gums. So you keep your eyes closed at the end. You don't want to see such a mouth up close. before the bite, before its oblivion in the goring of your soft parts, the speckled lips will curl back in a whinny of excitement. You just know it.

mystes posted:

You can also just use MAUI. Hybrid apps are specifically for when you want to use MAUI to present a webview control for your blazor app

That's not to say I think MAUI is good or anything but the whole idea of hybrid apps is to use MAUI to do something like Cordova for blazor apps but within the MAUI / .net ecosystem and within that context the idea probably more or less makes sense.
Yeah, I know. I'm specifically wondering if I should do a hybrid app because it's likely that Microsoft will put more pressure on in the future to turn everything into Blazor -- because, like I said, I can see the benefit of literally everything being one codebase and they'll just compile it differently for us. If they're gonna, I'll go with it, but if they're not gonna, I really don't want to.

mystes
May 31, 2006

CapnAndy posted:

Yeah, I know. I'm specifically wondering if I should do a hybrid app because it's likely that Microsoft will put more pressure on in the future to turn everything into Blazor -- because, like I said, I can see the benefit of literally everything being one codebase and they'll just compile it differently for us. If they're gonna, I'll go with it, but if they're not gonna, I really don't want to.
Imo, for better or worse, there's no way Microsoft is going to commit to a single approach hard enough for that to be an issue.

Jen heir rick
Aug 4, 2004
when a woman says something's not funny, you better not laugh your ass off

CapnAndy posted:

Yeah, I know. I'm specifically wondering if I should do a hybrid app because it's likely that Microsoft will put more pressure on in the future to turn everything into Blazor -- because, like I said, I can see the benefit of literally everything being one codebase and they'll just compile it differently for us. If they're gonna, I'll go with it, but if they're not gonna, I really don't want to.

I hate writing web apps and wish I could write native apps, but it seems like the world is moving to hybrid/web type apps. The economics are just too good. You can have a single team write an app that works on android iOS, macOS, Linux, and windows. You just can't beat that with native apps. You need a team or developer for each platform. The suits just aren't gonna pay for that. Even if the result is a shittier experience. People will get over it. poo poo sucks yo.

ChocolatePancake
Feb 25, 2007
If your old app is Xamarin I think upgrading it to Maui is the easiest path forward for sure.
Otherwise anything else you are looking at basically a total rewrite.

Adbot
ADBOT LOVES YOU

CapnAndy
Feb 27, 2004

Some teeth long for ripping, gleaming wet from black dog gums. So you keep your eyes closed at the end. You don't want to see such a mouth up close. before the bite, before its oblivion in the goring of your soft parts, the speckled lips will curl back in a whinny of excitement. You just know it.
Oh, we're doing Maui no question. I'm just looking ahead. Basically, if in five years Microsoft is gonna go "okay, everyone start using Blazor or whatever web language we've replaced it with for everything", I want to have spent those five years getting good at web development.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply