Attention, Internet Explorer UserAnnouncement: Jive has discontinued support for Internet Explorer 7 and below.In order to provide the best platform for continued innovation, Jive no longer supports Internet Explorer 7.Jive will not function with this version of Internet Explorer. Please consider upgrading to a more recent version of Internet Explorer, or trying another browser such as Firefox, Safari, or Google Chrome.(Please remember to honor your company's IT policies before installing new software!).
Hi Listers.We are experiencing intermittent timeouts with the ARS. Without medoing anything, the AR system becomes normal again after about 5 minutes. Hi Eric,Couple things you might check.Have you checked the indexing against the Run If in the escalations? NULL in the Run If ignores indexing and should be avoided.If you have a time calculation is the field on one side and the calculation on the other (Create Date. From: Action Request System discussion list(ARSList) mailto:On Behalf Of ZHANG, ERIC LSent: Thursday, January 20, 2011 4:11 PMTo:Subject: Strange ARS Timeout Problem.Hi Listers.We are experiencing intermittent timeouts with the ARS. Without me doing anything, the AR system becomes normal again after about 5 minutes.Eric,We had a similar symptom many yearsback. There were 3 DNS servers configured for our AR System server.Over time the first and second ones were retired and the DNS configurationdid not get updated.
So, for every DNS call the system had to waitfor the first and second servers to timeout before trying the third andif the third was busy everything just went to sleep while waiting for aresponse. We updated our DNS config and hosts file and everythingreturned to normal.Suppose there might also be other resourcesbesides DNS servers that could cause the same symptom. Our networkguys sniffed the network to see what we were waiting on.HTH,Dennis'ZHANG, ERIC L'Sent by: 'Action Request Systemdiscussion list(ARSList)' 03:10 PM. Subject Strange ARS Timeout Problem.Hi Listers.We are experiencing intermittenttimeouts with the ARS. Without me doing anything, the AR system becomesnormal again after about 5 minutes. From: Action RequestSystem discussion list(ARSList) mailto: On Behalf Of ZHANG,ERIC LSent: Thursday, January 20, 2011 2:11 PMTo:Subject: Strange ARS Timeout Problem.Hi Listers.We are experiencing intermittent timeouts with the ARS.Without me doing anything, the AR system becomes normal again after about 5minutes.
Thanks, Mark.I did go through the escalations that were running during the timeoutand couldn’t find anything out of the ordinary. The escalation logshows that all the escalation were completed in a fraction of a second and nodelays are showed in the sql log either.Eric-Original Message-From: Brittain, Markmailto:Sent: Thursday,January 20, 2011 3:30 PMSubject: Re: Strange ARS TimeoutProblemHi Eric,Couple things you might check.Have you checked the indexingagainst the Run If in the escalations? NULL in the Run If ignores indexing andshould be avoided.If you have a time calculation isthe field on one side and the calculation on the other (Create Date. From: ActionRequest System discussion list(ARSList) mailto: On Behalf Of ZHANG, ERIC LSent: Thursday,January 20, 2011 4:11 PMTo:Subject: Strange ARS TimeoutProblem.Hi Listers.We are experiencing intermittent timeouts with theARS.
Without me doing anything, the AR system becomes normal again after about5 minutes. Dennis,I have been trying to get our network guys to set up sniffer to monitorthe network traffic to/from ARServer. I will pass the DNS info to them.Thanks,Eric-Original Message-From: Dennis Rublemailto:Sent: Thursday,January 20, 2011 3:32 PMSubject: Re: Strange ARS TimeoutProblem.Eric,Wehad a similar symptom many years back. There were 3 DNS serversconfigured for our AR System server.
Over time the first and second oneswere retired and the DNS configuration did not get updated. So, for everyDNS call the system had to wait for the first and second servers to timeoutbefore trying the third and if the third was busy everything just went to sleepwhile waiting for a response.
We updated our DNS config and hosts fileand everything returned to normal.Supposethere might also be other resources besides DNS servers that could cause thesame symptom. Our network guys sniffed the network to see what we werewaiting on.HTH,Dennis'ZHANG, ERIC L'Sentby: 'Action Request System discussion list(ARSList)' 03: 10 PMPleaserespond toToccSubjectStrange ARS Timeout Problem.Hi Listers.We are experiencing intermittent timeouts with the ARS. Without me doinganything, the AR system becomes normal again after about 5 minutes. The DBA said there were no locks and blocks during timeouts. –Thanks,Eric-Original Message-From: LJ LongWingmailto:Sent: Thursday,January 20, 2011 3:40 PMSubject: Re: Strange ARS TimeoutProblemEric,Didyour DBA look for any locking? We used to experience this a lot till wefigure out what was happening.
I could give you SQLServer instructions onhow to find itbut you aren’t using thatand your DBA shouldbe able to run a query that’ll tell you about things that are causingblocking and things that are blocked. From: ActionRequest System discussion list(ARSList) mailto: On Behalf Of ZHANG, ERIC LSent: Thursday,January 20, 2011 2:11 PMTo:Subject: Strange ARS TimeoutProblem.Hi Listers.We are experiencing intermittent timeouts with theARS.
Without me doing anything, the AR system becomes normal again after about5 minutes. Interesting you asked about it. When wefirst encountered the problem, the Cursor Sharing was already set to FORCE bothin the Oracle database and ar.conf. The DBA then changed it to be EXACT inthe database while we kept it FORCE in ar.conf.
This change has improved theperformance noticeably, especially for the user to refresh Assigned Work table onthe Incident Management console. But the change didn’t eliminate the timeoutproblem.-Original Message-From: patchskmailto:Sent: Sunday, January23, 2011 1:08 PMSubject: Re: Strange ARS TimeoutProblem. What is the value for cursor sharing at the dblevel and on ar.conf file. Try to make it to Forced and see if it fixes thisissue. attend WWRUG11 www.wwrug.com ARSlist: 'Where the AnswersAre'attend WWRUG11 www.wwrug.com ARSlist: 'Where the Answers Are'.
Hi Listers.We are experiencing intermittent timeouts with the ARS. Without medoing anything, the AR system becomes normal again after about 5 minutes.
HiListers.Weare experiencing intermittent timeouts with the ARS. Without me doing anything,the AR system becomes normal again after about 5 minutes. We have sent BMC tech support all thelogs including api, filter, sql, escalation, thread, plug-in, arfork, even pstackoutput that were taken during hanging, and so far they haven’t been ableto identify the cause of the problem.-Original Message-From: Axtonmailto:Sent: Monday, January 24, 20115:45 PMSubject: Re: Strange ARS TimeoutProblem.
Try to get the api, filter, and sql logs leading upto the point where it started hanging. Those are your best indicator.Also check the arerror.log for crashes.
Hi Listers.We are experiencing intermittent timeouts with theARS. Without me doing anything, the AR system becomes normal again after about5 minutes. Hi Listers.We are experiencing intermittent timeouts with theARS. Without me doing anything, the AR system becomes normal again after about5 minutes. Yes, I did initial log analysis. As I saidin the original posting, there was 5-minutes gap in the api log, while nogap/waiting/error/long operation was showing in the sql log and escalation log.All the sql queries were for user ARESCALATOR in the sql log.-Original Message-From: Axtonmailto:Sent: Wednesday,January 26, 2011 8:18 AMSubject: Re: Strange ARS TimeoutProblem.
What do the logs say? Ihaven't seen that you've done analysis with the logs. Is there a gap intime in the logs (indicating the server was not doing anything)?
Is thereare gap in time in the logs (indicating a long operation was running? Hi Listers.We are experiencing intermittent timeouts with theARS. Without me doing anything, the AR system becomes normal again after about5 minutes. Ok.I just completely re-read the original post.allindications save one are that during that 5 minute interval the applicationserver lost connectivity with the DB server.
The only exception to thatappears to be the escalation thread which continued processing during that 5minute window.so, what I would do would be to setup a cron to run every30 seconds or every minute, something along those lines that issues a tracertbetween your remedy server and your db server. My primary thought is thatyou are losing network connectivity.even though the escalation server isstill workingit’s at least something you can try and report back. From: Action RequestSystem discussion list(ARSList) mailto: On Behalf Of ZHANG,ERIC LSent: Wednesday, January 26, 2011 7:19 PMTo:Subject: Re: Strange ARS Timeout Problem.Yes, I did initial log analysis. As I said in the original posting,there was 5-minutes gap in the api log, while no gap/waiting/error/longoperation was showing in the sql log and escalation log. All the sql querieswere for user ARESCALATOR in the sql log.-Original Message-From: Axton mailto:Sent: Wednesday, January 26, 2011 8:18 AMSubject: Re: Strange ARS Timeout Problem.
Whatdo the logs say? I haven't seen that you've done analysis with the logs.Is there a gap in time in the logs (indicating the server was not doinganything)? Is there are gap in time in the logs (indicating a longoperation was running?
Arerr 92 Timeout During Database Update 2016
Hi Listers.We are experiencing intermittenttimeouts with the ARS. Without me doing anything, the AR system becomes normalagain after about 5 minutes. I just put a cron job on thears server that runs traceroute every minute and appends the outputto an output file. Waiting for the next timeout.-Original Message-From: LJ LongWingmailto:Sent: Thursday,January 27, 2011 9:18 AMSubject: Re: Strange ARS TimeoutProblemOk.Ijust completely re-read the original post.all indications save one arethat during that 5 minute interval the application server lost connectivitywith the DB server.
The only exception to that appears to be theescalation thread which continued processing during that 5 minutewindow.so, what I would do would be to setup a cron to run every 30seconds or every minute, something along those lines that issues a tracertbetween your remedy server and your db server. My primary thought is thatyou are losing network connectivity.even though the escalation server isstill workingit’s at least something you can try and report back. From: ActionRequest System discussion list(ARSList) mailto: On Behalf Of ZHANG, ERIC LSent: Wednesday,January 26, 2011 7:19 PMTo:Subject: Re: Strange ARS TimeoutProblem.Yes, I did initial loganalysis. As I said in the original posting, there was 5-minutes gap in the apilog, while no gap/waiting/error/long operation was showing in the sql log andescalation log. All the sql queries were for user ARESCALATOR in the sql log.-Original Message-From: Axton mailto:Sent: Wednesday,January 26, 2011 8:18 AMSubject: Re: Strange ARS TimeoutProblem. What do the logs say?
Ihaven't seen that you've done analysis with the logs. Is there a gap intime in the logs (indicating the server was not doing anything)? Is thereare gap in time in the logs (indicating a long operation was running? Hi Listers.We are experiencing intermittent timeouts with theARS. Without me doing anything, the AR system becomes normal again after about5 minutes.
Ok.Ijust completely re-read the original post.all indications save one arethat during that 5 minute interval the application server lost connectivitywith the DB server. The only exception to that appears to be theescalation thread which continued processing during that 5 minutewindow.so, what I would do would be to setup a cron to run every 30seconds or every minute, something along those lines that issues a tracertbetween your remedy server and your db server. My primary thought is thatyou are losing network connectivity.even though the escalation server isstill workingit’s at least something you can try and report back. Yes, I did initial loganalysis.
As I said in the original posting, there was 5-minutes gap in the apilog, while no gap/waiting/error/long operation was showing in the sql log andescalation log. All the sql queries were for user ARESCALATOR in the sql log.-Original Message-From: Axton mailto:Sent: Wednesday,January 26, 2011 8:18 AMSubject: Re: Strange ARS TimeoutProblem. What do the logs say? Ihaven't seen that you've done analysis with the logs. Is there a gap intime in the logs (indicating the server was not doing anything)?
Is thereare gap in time in the logs (indicating a long operation was running? Hi Listers.We are experiencing intermittent timeouts with theARS. Without me doing anything, the AR system becomes normal again after about5 minutes. Subject Re: Strange ARS Timeout Problem.Good idea.
I just puta cron job on the ars server that runs traceroute everyminute and appends the output to an output file. Waiting for the next timeout.-Original Message-From: LJ LongWing mailto:Sent: Thursday, January 27, 2011 9:18 AMSubject: Re: Strange ARS Timeout ProblemOk.I just completely re-readthe original post.all indications save one are that during that 5 minuteinterval the application server lost connectivity with the DB server. Theonly exception to that appears to be the escalation thread which continuedprocessing during that 5 minute window.so, what I would do would beto setup a cron to run every 30 seconds or every minute, something alongthose lines that issues a tracert between your remedy server and your dbserver.
My primary thought is that you are losing network connectivity.eventhough the escalation server is still workingit’s at least somethingyou can try and report back.From: Action Request System discussionlist(ARSList) mailto: On Behalf Of ZHANG, ERICLSent: Wednesday, January 26, 2011 7:19 PMTo:Subject: Re: Strange ARS Timeout Problem.Yes, I did initial log analysis.As I said in the original posting, there was 5-minutes gap in the api log,while no gap/waiting/error/long operation was showing in the sql log andescalation log. All the sql queries were for user ARESCALATOR in the sqllog.-Original Message-From: Axton mailto:Sent: Wednesday, January 26, 2011 8:18 AMSubject: Re: Strange ARS Timeout Problem. Whatdo the logs say? I haven't seen that you've done analysis with thelogs. Is there a gap in time in the logs (indicating the server wasnot doing anything)? Is there are gap in time in the logs (indicatinga long operation was running?On Tue, Jan 25, 2011 at 5:49 PM,ZHANG, ERIC L wrote:.We have sent BMC tech supportall the logs including api, filter, sql, escalation, thread, plug-in, arfork,even pstack output that were taken during hanging, and so far they haven’tbeen able to identify the cause of the problem.-Original Message-From: Axton mailto: Sent: Monday, January 24, 2011 5:45 PMSubject: Re: Strange ARS Timeout Problem. Tryto get the api, filter, and sql logs leading up to the point where it startedhanging.
Those are your best indicator. Also check the arerror.logfor crashes.There are things that can cause behaviorlike this that the logs will indicate. For example, try creatinga computed group during production operations, or importing a deployableapplication.On Thu, Jan 20, 2011 at 3:10 PM,ZHANG, ERIC L wrote:.Hi Listers.We are experiencing intermittenttimeouts with the ARS.
Arerr 92 Timeout During Database Update Download
Without me doing anything, the AR system becomesnormal again after about 5 minutes. Not sure if this can help.We had the same issue as system used to give intermediatetimeout issue and only similarity was the time out used to happen at a fixedtime always.Is this the case with you also???We found out that some views were running @ the specifiedtime and system used to go for a full scan on the major form. No looks we foundbut yes the system performance used to go down.As Axton said if there is any issue with the network theyou can see that in the error log itself.Not ignoring the network issue try changing the entry inthe ORA file from hostname to IP. We use it in order to minimize the issue incase if there is any DNS issue also.One more thing you need to find out is the issue happeningthrough Midtier only or From User Client or both. From: Action RequestSystem discussion list(ARSList) mailto: On Behalf Of Dennis RubleSent: Friday, January 28, 20114:08 AMTo:Subject: Re: Strange ARS TimeoutProblem.Eric,Youmight add a nslookup command to your cron job to see if a dns lookup isfailing.
Arerr 93
A dns failure will give the same ARS symptom as a network outagebecause it is an operation that the server must complete before communicationscan happen.Goodluck,Dennis'ZHANG, ERIC L'Sentby: 'Action Request System discussion list(ARSList)' 04:26 PMPleaserespond toToccSubjectRe: Strange ARS Timeout Problem.Good idea. I just put a cron job on the arsserver that runs traceroute every minute and appends theoutput to an output file. Waiting for the next timeout.-OriginalMessage-From: LJ LongWing mailto:Sent: Thursday, January 27, 2011 9:18 AMSubject: Re: Strange ARS Timeout ProblemOk.I just completely re-read the originalpost.all indications save one are that during that 5 minute interval theapplication server lost connectivity with the DB server. The onlyexception to that appears to be the escalation thread which continued processingduring that 5 minute window.so, what I would do would be to setup a cron torun every 30 seconds or every minute, something along those lines that issues atracert between your remedy server and your db server. My primary thoughtis that you are losing network connectivity.even though the escalation serveris still workingit’s at least something you can try and report back.From: Action Request System discussionlist(ARSList) mailto: OnBehalf Of ZHANG, ERIC LSent: Wednesday, January 26, 2011 7:19 PMTo:Subject: Re: Strange ARS Timeout Problem.Yes, I did initial log analysis. As I said in the originalposting, there was 5-minutes gap in the api log, while nogap/waiting/error/long operation was showing in the sql log and escalation log.All the sql queries were for user ARESCALATOR in the sql log.-OriginalMessage-From: Axton mailto:Sent: Wednesday, January 26, 2011 8:18 AMSubject: Re: Strange ARS Timeout Problem. What do the logs say?I haven't seen that you've done analysis with the logs.