Dell VC Plugin – iDRAC Queued Jobs

I’ve been doing some standard maintenance on my Dell R815 hosts (upgrading BIOS, firmware, drivers etc.) and ran into some difficulty with one particular server. I use the Dell Management Plugin for VMware vCenter to carry out these operations, and besides being slow it works well.

I simply run the firmware upgrade wizard which stages the current versions to the iDRAC and schedules the deployment job to execute on boot using UEFI system services. The process automatically enters the host into maintenance mode and reboots to start the UEFI jobs.

On this particular server, the job failed and I was left in a situation where the job was scheduled, would not execute and I couldn’t see any option to delete these jobs. In my case it was a BIOS update to address the PSOD issue affecting AMD Opteron 62xx series processors.

I had a look through the logs and found the following entries (also shown in the vCenter tasks);

      • [Firmware Update] File: R815_BIOS_F8FCX_WN32_3.0.5.EXE – Status: Failed – Message: iDRAC – A job for this update already exists in the queue and prevents the new job from being scheduled.
      • [Firmware Update] File: R815_BIOS_F8FCX_WN32_3.0.5.EXE, Status: Failed, iDRAC:, Host, Recommended resolution actions: iDRAC – Delete the existing job before scheduling a new update job. For Remote Services API consumers, enumerate all the jobs to find and identify the duplicate job. Then invoke the DeleteJob() method to delete this job and enable the new update job to be scheduled
      • [Firmware Update] The state of update job for file (R815_BIOS_F8FCX_WN32_3.0.5.EXE) has changed from (NEW) state to (FAILED) state.

The solution is pretty simple looking at these log entries, delete the existing job and then run the wizard again. The only problem is that there is no obvious way to delete the jobs.

To speed up the resolution I logged a support call with Dell who wanted me to reset the iDRAC to factory defaults and send them various logs (which sounded like they were clutching at straws) so I started doing my own research and found a solution using WinRM nested deep in one the Dell support forums.

Windows Remote Management (WinRM) is the Microsoft implementation of the WS-Management protocol which provides a secure way to communicate with local and remote computers using web services.

Remember to replace the $USERNAME, $PASSWORD and $IPADDRESS with your own values.

Check for existing jobs;
## Use WinRM to check for existing Jobs
winrm e cimv2/root/dcim/DCIM_LifecycleJob -u:$USERNAME -p:$PASSWORD -SkipCNcheck -SkipCAcheck -r:https://$IPADDRESS/wsman:443 -auth:basic -encoding:utf-8
InstanceID = JID_001371642178
JobStartTime = TIME_NA
JobStatus = Failed
JobUntilTime = TIME_NA
Message = Task for this device is already present.
MessageID = RED014
Name = update:DCIM:INSTALLED:NONPCI:159:3.0.4
PercentComplete = NA

** I had multiple entries but have only shown an excerpt to keep the post short.

Delete all existing jobs;
## Use WinRM to delete all jobs
winrm invoke DeleteJobQueue "cimv2/root/dcim/DCIM_JobService?CreationClassName=DCIM_JobService+Name=JobService+SystemName=Idrac+SystemCreationClassName=DCIM_ComputerSystem" @{JobID="JID_CLEARALL"} -r:https://$IPADDRESS/wsman -u:$USERNAME -p:$PASSWORD -SkipCNcheck -SkipCAcheck -encoding:utf-8 -a:basic -format:pretty
<n1:deletejobqueue_output xml:lang="" xmlns:n1="<a href=" http:="""" wbem="" wscim="" 1="" cim-schema="" 2="" root="" dcim="" dcim_jobservice"="">"&gt;
<n1:message>The specified job was deleted</n1:message>
Check for existing jobs (again);
## Use WinRM to check for existing Jobs
winrm e cimv2/root/dcim/DCIM_LifecycleJob -u:$USERNAME -p:$PASSWORD -SkipCNcheck -SkipCAcheck -r:https://$IPADDRESS/wsman:443 -auth:basic -encoding:utf-8
JobStartTime = TIME_NA
JobStatus = Pending
JobUntilTime = TIME_NA
Message = NA
MessageID = NA
PercentComplete = 0

So it’s easy when you know how, but this issue still managed to waste a lot of my time! Hopefully my post will save you from a similar experience.

 35,002 total views,  1 views today

Author: Jon Munday

An independent IT contractor with a strong focus on VMware virtualisation and infrastructure operations. I am inspired by technology, not afraid to question the status quo and balance my professional commitments with entertaining my three awesome kids (Ashton, Oliver and Lara).

10 thoughts on “Dell VC Plugin – iDRAC Queued Jobs”

  1. Hi, thanks for your posting.
    I have the same problem with an R510.
    The DELL VC plugin starts working on the job, but then gives me the error messages.

    I tried your commands, checking for existing jobs works fine – but I receive an error at the point “delete all existing jobs”:
    — Error: Invalid use of command line. Type “winrm -?” for help. —

    The command line is correct, I verified it with “Lifecycle_Controller_2_0_Web_Services_Interface_Guide_for_Windows_v1.1.pdf”

    It’s an iDRAC 6 with FW 1.95.

    Thanks in advance,

  2. Hi Chris,

    What version of PowerShell do you have installed? Perhaps you could try adding a set of single quotes around this section of the code ‘@{JobID=”JID_CLEARALL”}’.

    So the full syntax will look like this;

    ## Use WinRM to delete all jobs
    winrm invoke DeleteJobQueue "cimv2/root/dcim/DCIM_JobService?CreationClassName=DCIM_JobService+Name=JobService+SystemName=Idrac+SystemCreationClassName=DCIM_ComputerSystem" '@{JobID="JID_CLEARALL"}' -r:https://$IPADDRESS/wsman -u:$USERNAME -p:$PASSWORD -SkipCNcheck -SkipCAcheck -encoding:utf-8 -a:basic -format:pretty

    I suspect the original code without the single quotes will work fine from a command prompt, but you will need the quotes when executing this from PowerShell.

    Let me know if this works?


    1. Thanks Jon for posting this and for the reply suggesting the single quotes. I’ve been trying to execute this via powershell on Win2k12 and had the same error.

      I’m trying to upgrade the firmware on 41 new Dell R630’s using OpenManage Essentials and after installing most of the upgrades, the Ethernet card firmware failed to install with the resulting error suggesting to remove the queued jobs.

      I have to admit, being a Linux admin, I’m new to PowerShell 🙂

      Thanks again.

  3. Hi Jon,
    it’s an Windows Server 2008R2 with Powershell 2.0.
    I didn’t try using the cmd shell, and by adding the set of single quotes it works fine from Powershell! It’s now possible to install / upgrade the new firmware files via DELL VC plugin on my R510.

    Thank you very much!
    Cheers from Germany,


Leave a Reply

Your email address will not be published. Required fields are marked *