Noticed that my Smartermail IIS logs had been getting huge these days so went to investigate.
It seems to be something to do with the Bandwidth Counting Scheduled task getting stuck in an infinite loop.
When I run it, the SCP Server keeps hitting the url “/services/svcDomainAdmin.asmx” continuously over and over (we’re talking 3gb a day of logs just showing that!)
Sm 15.7 (not yet upgraded to 16)
Could it not do both? If it’s a new account set the date as today (or yesterday?) but on an import, like you said, maybe put it a year back (I think that would cover most users!)
Ok i see what you mean that there is a loop that looks at the date starting with 1/1/0001 and then increases this value by 1 day before doing the API call again.
It looks like alot of the providers do the same but they are generally checking files and not making an API call.
I think the easy solution for all is that when the package is added we set a BandwidthUpdated value. The question is should we make this date the day the package is created or something more generic like a year before to cover any imported packages?
The start date (in the $POST SOAP value call to Smartermail svcDomainAdmin.asmx) isn’t blank, it’s 0000-00-01 and then increments over and over.
Sadly I have deleted all the traces now or I could send them to you to see.
The question is – if the bandwidth collection on an email only package hasn’t been run before, which date do you send as the StartDate?
We make the GetDomainStatistics API Call to Smartermail with the startdate and EndDate.
Would you be able to make the API call for us against your Smartermail server to see what happens if the start date is blank?
6 minutes is for the bandwidth to be collected on all hosting spaces (As a test server it has alot of different spaces and providers). If this acts as you explain i would expect it to have a large different between the first and second time it is ran as the second time the BandwidthUpdated has a value.
That still wouldn’t explain why the “since” value starts at 0000-00-00. (if you capture all the requests, you can see this in the SOAP call)
I was looking through the source code and I can’t find anything which actually checks the “since” date is valid.
6 minutes to go from 0000-00-01 up to 2019-03-12 seems about right.
The bandwidth task on our test environment takes 6minutes on first run when the value is null. This is reasonable and once i ran it a second time after confirming the value is filled it took around the same amount of time.
I wonder if this is the call on the Smartermail API which times out due to a large amount of data it may have to collected.
It would be interesting to see if this is the case and we can look at reporting this to Smartermail.
I’ll check that later with a new package.
However it doesn’t explain the original issue (which is the SM being hit in a loop).
I traced all of the calls being made, and as I wrote above, the issue is the SOAP request to svcDomainAdmin.asmx is passing 0000-00-01 as its first item (I assume that is NULL) and then going up from there, 0000-00-02, 0000-00-03, etc, etc – hence the seemingly endless loop.
I can confirm new packages do have a NULL BandwidthUpdated value but if i run the bandwidth task this completes and the value is filled.
The domains you are seeing with NULL and causing an issue are these older domains? I’m wondering if part of calling the Smartermail API its returning alot of data and either Smartermail or SolidCP is failing with the call.
Could you try make a new hosting space with Smartmail to see if there is an issue with the report when it next runs and if the value is filled?
OMG, finally, I have found the issue!
It’s due to the “BandwidthUpdated” field being NULL for a package in the dbo.Packages table.
So in the calculate bandwidth procedure, SolidCP will count up from the last updated date until today, hitting Smartermail each time to get the data.
If the field is NULL however (which is the default for new packages) – then 0000-01-01 will be used as the start date!?! So the counter goes into a loop thousands of times!!!
A quick fix to get the process running again for now is to run the following SQL command:
SET BandwidthUpdated = ‘2019-01-01’
WHERE BandwidthUpdated is null
But I assume that any new packages added will also have the same issue?
Nothing in either of those logs 🙁 (the audit log is just the one under root right – Default.aspx?pid=AuditLog&UserID=1)
So there is no way to get a verbose log of what a task is actually doing?
It works for random packages, but not others – with nothing seemingly linking the working ones together.
- Views3768 times
- Answers24 answers