Opinion: What makes a good asp.net app??

[eramgarden]

At this new job of mine, I'm working with a asp.net application that was done by another company...

 

1.# of users..maybe 200

2. It went Live, 5 months ago

 

I'm new at this job but some stuff I've noticed:

1. They keep datasets in cache (they used to keep them in session). Users get "failed to load page from cache" from time to time

2. We got "out of memory exception" 5 or 6 times last week

3. Developers "throw" the errors instead of trapping them. Users get asp.net pink error pages

4. The code is layered into "facade", data layer and then the presentation layer.

5. Everything is in stored procs...maybe 1070 stored procs.

 

now I told my managers that this is the slowest and buggiest application I've ever worked with. They defended the other company!.

 

My questions

What is a good asp.net application? shouldnt an ASP.net app support 10,000 users (someone told me that) without cache and memory issues?? And i think the app has many bugs but the manager didnt like it when i said it.

[PlausiblyDamp]

1. I've personally no issues with keeping read only data in cache , however if the item isn't in cache that is not an excuse to give up - go reload the data into cache...

however if the data being cached is excessively large (possible considering your point 2) then I would be either more selective in what I cache or more aggresive in ageing it out.

2. CLRProfiler is your friend. Use it. If you are freeing up unmananged resources correctly and not blindly stuffing anything and everything into session / application variables then you shouldn't be running out of memory.

3. Components can throw exceptions - it is a standard .Net way of signaling an error condition. Dr Watson is not an acceptable way to convey an error message under windows why should it be acceptable for a web application to die in a similar manner? At a minimum it should be directing them to a 'prettier' error screen. Deliberately throwing exceptions to crash the app is piss poor practice to say the least.

4. Nothing bad about the architecture, implementation may be another issue...

5. As a rule I am a fan of stored procs. Implemented well they work fine. 1070 may not be an excessive number although if there is a lot of redundancy then parameterising them better may be a consideration for the future.

 

.Net will support well above 200 users without anywhere as many problems as you are getting as long as the design / code / architecture is of a reasonable standard.

[samsmithnz]

1. I don't know much about Caching, but it sounds like it's expiring too quickly. I would question why these datasets are being kept in the cache. Is it a list of offices or something that rarely changes? (the correct way to use the cache), or is it a more dynamic list (such as maybe all customers who live in region 4 and own more than 5 thingy-bobs). (wrong way to use Caching)

 

2. You shouldn't be getting out of memory errors. We had a simliar problem, called up Microsoft and found an infinite loop in a stored procedure that was causing an overflow that throws a SQL error - BUT NOT an exception. I don't know what sort of crack MS was smoking when they made that decision. It's a pain having to check the SQL errors collection. The other thing MS found was that we were appending strings together (strString1 & strString2), and it's much better and easier on the .NET Framework if you use the StringBuilder type to create strings.

 

3. It's ok to throw exceptions, but only if you need to. You should never see "throw ex" just for the hell of it though, an unhandled exception will naturally flow up to the calling function and then you can catch the error at the top of the stack.

 

4. I agree with PlausiblyDamp, architecture sounds fine.

 

5. Stored procedures are the way to go, they precompile and are faster.

 

We just finished a very large ASP.NET project (16,000 lines), and we brought a MS guy in to audit our work and he told us some very interesting things that helped us to double the speed of our app. I'd recommend this book for a start: http://www.amazon.com/exec/obidos/tg/detail/-/0735618518/qid=1121867632/sr=8-1/ref=pd_bbs_1/102-5552162-2846554?v=glance&s=books&n=507846

 

It's also available online in MSDN if you search for it, and it's the best book out there for optimizing your .NET applications.

[eramgarden]

#2..they said there's a leak in "Enterprise Library" and they got a patch from Microsoft's site. This is part of it;

The Enterprise Library team recently became aware of a memory leak issue in the Configuration Application Block, which is used by all other blocks in Enterprise Library. This zip file contains a fix to the issue.

 

I did apply it to the production but we still got the error. For example yesterday, memory consumption reached 695Mb and no one could access the app. They changed the cache setting in web.config to 1200. Any thoughts on this?

 

for #1 So that error happens when data is lost in cache??

 

for #4 How can I identify the bad implementations? is there a website about good Dotnet practices that I can compare with what they've done?

 

last question Someone in the meetings said that lots of companies who use dotnet have these issues! I dont believe that and they said with the next release of dotnet, most of these memory and cache issues will be resolve. Any thoughts on that?

[samsmithnz]

There are no memory issues in .NET. You shouldn't be getting memory leaks and if you do, increasing a value in web.config isn't going to help you find it.

[PlausiblyDamp]

1. If you try to retrieve an expired item from cache it just returns null - this is something that should be handled by the application logic; I doubt 'giving up' and just telling the user they couldn't find it in cache is the correct response.

 

4. You get unhandled exceptions and excessive memory usage resulting in crashes :D

 

2. I would seriously check the running app with the CLRProfiler - although .Net memory management will free up unused objects it isn't a magic wand that prevents all memory problems. If the app is stuffing things in application variables and never deleting them memory can leak etc.

 

In regards to the comment

lots of companies who use dotnet have these issues!

that could possibly reflect on the fact lots of companies have written poor quality software in .Net rather than any issues with .Net itself ;) Edited November 16, 2005 by PlausiblyDamp

[eramgarden]

2 last questions, then I'm going to look into the Book and CLRProfiler..

 

Love your #4 answer :D I mentioned that to the manager yesterday and he defended the app like he wrote it or something! I call it misplaced loyalty.. :)

 

Anyway-

 

1. "There are no memory issues in .NET".. do you mean that .net takes care if memory management and garbage collection by itself UNLESS (as PD mentioned) app is stuffing things in application vars and never deleting them?

 

2. Users want to keep their session for like 4 or 5 hours (maybe even the entire day)..That's why datasets are held in Cache and "cache expiration" value is increased from 10 min to 20 mins. (This increase caused memory error :D ). What's the solution to giving the users the option to keep their session open all day?

[PlausiblyDamp]

1. Garbage collection will reclaim memory from unused (i.e. aren't referenced anymore) objects - if there is a reference somewhere (session, application, cache etc) then the garbage collection will be unable to reclaim the resources.

 

2. Why would they need a session that lasts 4 or 5 hours :confused: What information is being stored in a session that needs to live for 5 hours???? More importantly what is meant by

Users want to keep their session for like 4 or 5 hours

and who even asked the users for their preference for server side code implementation?

 

It might help if you gave a little more detail about the type of information being stored in Cache etc and why the need for it to be available for 5 hours at a stretch. If though through you will probably find there is an alternate solution , possible storing just IDs or similar in Session and requerying the DB when information is required.

 

It sounds very much like a 'thick client' mentality has been applied to what is essentially a 'thin client' application. who would have though it could cause problems?

[eramgarden]

#2. The orig system was done in VAX. Users had issues with that because they couldnt keep their sessions for long period of time.

 

The company that developed the app told them it's doable and it's going to be one of the features!

[eramgarden]

I ordered the book...Thanks..

 

The company who did the app for us is looking into "hardware" issues and buying more memory to solve the "out of memory error".

 

Based on what you guys said..this isnt hardware related and we should look at the code/implementation...Anyway, before I open my mouth and say something to my manager ..this is the hardware config we have right now..

 

would buying hardware fix the "out of memory" make any diff??

 

--

Web server

Model: Dell PowerEdge 2600

CPU: 2 x 2.0 GHz Intel Xeon, Hyperthreading ON

RAM: 2Gb

HDD: 2 SCSI channels, RAID1 16GB and RAID5 33GB, respectively

OS: Windows 2000 Server, SP4

 

SQL Server

Model: Dell PowerEdge 6650

CPU: 4 x 2.7GHz Intel Xeon, Hyperthreading OFF(?)

RAM: 4Gb

HDD: 3 RAID0 arrays, separating tempdb, application data, and transaction logs

OS: Windows 2000 Server, SP4

[PlausiblyDamp]

I doubt seriously that upgrading the hardware will fix these problems, increasing the ram may delay the problem but it probably won't fix a thing.

Still not entirely sure why sessions need to be kept for so long. What exactly do they mean by session in this context as I get a feeling it is different to an ASP.Net session. Could you give an example of what day to dy functionality requires server side session management for 5 hours plus.

[samsmithnz]

I agree. Why do you need those datasets in cached sessions? They don't sound like they belong... I'd be willing to bet that if you took out the caching it would probably go faster...

 

The hardware looks fine, you need to make it clear that it's not fixing the problem. How long does it take to make an out of memory error? Minutes? Hours? Days? If it's longer rather than shorter then it's definitly a memory leak.

 

Also, are you using COM objects in your ASP.NET project? If you are this could be the cause of your problem. I have some good recommendations there.

[eramgarden]

No COM objects. Since I'm new at this job, the team only listens to what the other company says(the company that developed the app for the company I'm working for)...

 

If they dont keep it in cache, what should they be doing. I know they tried Session

 

The other company asked us to move to "Windows 2003" and they also asked us to do this :

 

We need to change the setting for memoryLimit in machine.config from 30 to 60 on the production web server. This will increase the amount of memory the asp worker process can consume before throwing an �out of memory� exception. The limit was changed to 30 in the mistaken belief that there was 4GB of memory on the server. For 2G of memory, the optimal setting is the default of 60.

 

Any thoughts on that? ( I really do believe what u guys say over what the other company is saying. I think they're just covering for themselves).

[samsmithnz]

But how long does it take to throw the out of memory error? It could be a server configuration error if it happens almost right away.

[samsmithnz]

Here's what you have to do though: You have to prove the company wrong. I use a lot of 3rd party vendors, and before we ever go to the company with an accusation that their part isn't working or that there is a bug in their product, try to find a way to prove or replicate the issue that they can replicate in their test environment. I know it's not as easy as it sounds, but if you can find where it's breaking, you can go back to them with "it's breaking here, fix it." - which is better than what is so far your opinion against theirs...

[PlausiblyDamp]

Could you not remove the need for all these objects to remain in memory for so long? There has got to be a better solution than making session state persist for such inordinate lengths of time.

[eramgarden]

Yeah, we just got "out of memory execption" again.

 

I cant tell if it happens right away or not but for example, we have a search page, calls a stored proc..User clicked on "search" button and got the out of memory. She waited for a min, tried it again and it worked fine.

 

Why they have the stuff in memory for so long? I dont know why they designed it this way. This is a financial/invoicing system. Users are doing something, get on the phone, come back 30 mins/1 hour later..they want their session/cache stuff there for them.

 

That Machine.Config change seems like didnt help either.

 

They might bring an outside person to do a code review

 

I do agree totally with you guys. I have started looking into CLRProfiler and will look at other 3rd party tools as well.

[eramgarden]

Someone even got "

 

Internal Connection Fatal Error

 

But some people were working fine, and some were getting memroy exception, this user got the error above.

 

This is the worst app I've worked with. What surprises me is that the developers who worked on it have all sorts of MicrosSoft certifications!

[eramgarden]

I forgot, we have Deadlocks too :D

[samsmithnz]

Yeah, we just got "out of memory execption" again.

 

I cant tell if it happens right away or not but for example, we have a search page, calls a stored proc..User clicked on "search" button and got the out of memory. She waited for a min, tried it again and it worked fine.

 

Why they have the stuff in memory for so long? I dont know why they designed it this way. This is a financial/invoicing system. Users are doing something, get on the phone, come back 30 mins/1 hour later..they want their session/cache stuff there for them.

 

That Machine.Config change seems like didnt help either.

 

They might bring an outside person to do a code review

 

I do agree totally with you guys. I have started looking into CLRProfiler and will look at other 3rd party tools as well.

 

Did you ask the searcher what they searched for? Maybe you can reproduce the error searching for stuff.

 

One other thing, it sounds like you're using inproc session state, which isn't very scalable. I wonder whether you should be using SQL Server session state which will support a much greater number of users and not suck up as much memory on the server. Check it out in MSDN here: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspp/html/aspnetsessionstate.asp

[eramgarden]

Thanks. Will check that too.

 

We did ask the user. She couldnt remember. As she reported the error, I went to the search page, clicked and got the out of memory error. Then other users started reporting it. I know that page calls a stored proc. Should I follow that lead?? Or do I need to know what caused THE FIRST out of memory error(ie, the first reported out of memory of the day) and follow that lead.

 

It was fixed when we rebooted the machine.

 

Let me also check the "in proc" session state.

[samsmithnz]

It's hard to tell what lead to follow. Because everyone was affected and a server reboot fixed it (although I'm sure an IISreset would do the same thing and be a lot quicker), it sounds like something else.

 

Have you monitored the aspnet process on the server? And start writing down the times of the out of memory errors, see if there's a pattern (every 4 hours, every 7 business hours, etc.)

[samsmithnz]

Although checking out the stored proc wouldn't hurt. See if there are any goto's in the stored proc. that could be one of your problems - an infinite loop that causes an overflow - which ASP.NET can't catch for some stupid reason.

[eramgarden]

They brought a guy to look at Garbage Collection stuff in the code...

 

Meanwhile, one of their developers said this:

 

we found evidence that upgrading to Windows 2003 would allow the ASPNet worker process to access even more memory, as much as 1.8Gb (assuming we had that much available on the machine for IIS, after the OS and other services started).

 

 

is this correct?