Note: This article covers material present in Version 1 Revision 3 and below.  Topics found below may be mitigated in the most current version of the ESXi 5 STIG.  Ensure you are using the most current version of the DISA STIG documents.

** Updated 9/9/13 to include distinction on Privilege Separation between v5.0 and 5.1 and above **

Note: All settings within the sshd_config file are case sensitive.  Options are typically all lower case {yes|no} and are all expressed as <settings> <option> within the configuration file.  The easiest way to verify an issue during your configuration is to have a second console open and run 'tail -f /var/log/auth.log' while attempting a connection.

I’m going to start with the SSH daemon findings.  Why?  Because it’s been a topic of conversation internally for the past few days, and it’s just littered with problems to talk about.

So, here’s the format we’re going to follow.  I’m going to divide this into sections calling out the STIG ID and Rule Title.  If the rule causes no issue I’ll call it out, but that will be about it.  By the end of the article you’ll have a list of rules that you comply with, or at the very least you’ll know why you cannot.  Here we go…

The SSH daemon settings in the ESXi 5 STIG pose a bit of a problem for those implementing them.  To make it worse, there’s some confusion and lack of direction to several of the findings, plus one permanent finding that you cannot set on 5.0, but can on 5.1+.  Let’s get started with the SSH daemon settings below.  I’ll cover the SSH client settings in the next post.

SRG-OS-000023-ESXI5 – SSH Daemon Logon Banner

I promise I didn’t start with a problem child on purpose, but here we go.  In ESXi 5 and beyond the default sshd_config that ships with the product specifies an alternate banner file.

#grep Banner /etc/ssh/sshd_config
Banner /etc/issue

Since the STIG calls for /etc/banner we already have a problem, but lets look at what can be done.  To start I want to explain the difference between /etc/banner, /etc/issue, and /etc/motd.  First off, there’s a trick to changing /etc/banner that involves the local.sh script run at boot.  Modifying the /etc/banner file normally, via vi, is not possible.  In order to do it you have to pre-populate the /etc/banner file at boot via the local.sh script.  That is the only way to make it work for this finding.  Something that’s not mentioned in the STIG, and something we will have to use later when I cover the ssh_config file.

Shortly after I published this article William Lam and I had a conversation about the ability to modify the /etc/banner file and its use in this finding.

If you are to use the /etc/banner file in order to be compliant with this finding you must build it at each boot via the local.sh file.  You can see how to do this over on William's blog at How to Enable SSH Security Banner on ESXi, toward the end of the article.  I will also be covering this method in the next article on SSH Client settings and the STIG, since we will have to use this method to configure the ssh_config file.

In the end you are left with a choice.  If you have an IA group that are sticklers for exact compliance to the letter of the STIG use Williams article above to create your banner on the fly.  However, the best choice, and the one VMware created specifically for login banners, is to use the /etc/issue file.  My comment to DISA will be to modify this finding to have the check/fix point at /etc/issue.

Now, /etc/issue and /etc/motd are modifiable by you and have two distinct usages.  The /etc/issue file allows you to display text prior to logging in.  On the ESXi Console this is displayed immediately after the /etc/banner, but prior to the Username prompt.  Coming from SSH the /etc/issue text is displayed prior to the Password prompt.  That means, in both instances, the /etc/issue file will display the DoD Consent banner appropriately, prior to login of the user.  Text placed in /etc/motd will display after the user logs in, both on the ESXi Console and via SSH.  You best choice is the /etc/issue file.

SRG-OS-000027-ESXI5 – SSH Daemon Single Session Limit

The only thing I want to point out here is that a standard openSSH configuration file is <keyword><space><value>  The fix text here leads one to believe it’s <keyword>=<value>.  So, you should make sure sshd_config has a line in it that says MaxSessions 1, not MaxSessions=1, if you want to keep with the format of a standard openSSH configuration file.

SRG-OS-000033-ESXI5 and SRG-OS-000112-ESXI5 – SSH Daemon Cryptography of Remote Access Sessions / SSHv2

These two are exactly the same.  Just check and make sure Protocol 2 is present in the /etc/ssh/sshd_config file.  It’s there by default from VMware and is also the default standard in OpenSSH.

SRG-OS-000109-ESXI5 – No Root Logon via SSH

Pretty easy, but I should point out that when you set this you will no longer be able to connect remotely via SSH until another user is created on the system.  Just make sure you have access to the DCUI console before setting this, if you’re working over SSH right now.  I recommend you set this at the very end.  Per the STIG (later) you’ll see that you cannot create local users that are not AD backed so let’s walk through that one later.

SRG-OS-000250-ESXI5 – SSH Daemon MACs Employing FIPS 140-2

It’s not so much that this check is wrong, just very limiting.  So the point here is to ensure the daemon is using FIPS 140-2 approved hashes.  Ok, no problem.  The trick is in the finding text.

If no lines are returned, or the returned MACs list contains any MAC other than hmac-sha1, this is a finding.

My rub here is that the check says that it absolutely has to say hmac-sha1 and no others, but openSSH has multiple FIPS 140-2 approved hashes available, including ones in the SHA2 family.  Why limit it to only hmac-sha1?

GEN005515-ESXI5-000100 – SSH Daemon Must Disallow TCP Forwarding

Finally, one that’s straight forward out of the box.  This one is important too because the default in openSSH is yes, so you must explicitly set this to no in the sshd_config file.

GEN005517-ESXI5-000101 – SSH Daemon Must Disallow Gateway Ports

This is more of a note for DISA than you, the implementor.  The check text says that if GatewayPorts is yes that this is a finding in the check text.  Fact is, GatewayPorts isn’t a boolean setting.  GatewayPorts can be {yes|no|clientspecified}.  So, technically you could configure this as ClientSpecified and comply with the STIG.  🙂  I’m making a joke, but if you want to be technical it’s completely true.  Just set GatewayPorts in the sshd_config file to no and move on.  The openSSH default, in case you are curious, is no as well.

GEN005519-ESXI5-000102 – SSH Daemon disallow X11 Forwarding

We ship ESXi 5 and above with X11Forwarding set to no in the sshd_config file by default.  No action is necessary on your part.

GEN005521-ESXI5-000103 – SSH Daemon Restrict Login to Specific Users / Groups

What they want you to do is look for the keyword “AllowGroups” in the sshd_config file.  The AllowGroups keyword tells openSSH to restrict SSH access to specific local groups based on a pattern match of the group name.  By default all groups are allowed access.  So you think, hey… that’s a good thing.  Let’s tighten up SSH and only allow a specific group or two to access via that method.  Ok, so lets look at how they have you fix it, and the differences between ESXi 5.0 and later.

The fix text for this finding says to modify the sshd_config file and add the AllowGroups keyword and you’re done.  Do you know what this really did to your system’s security?  ABSOLUTELY NOTHING!  That’s right, unless you give it names to match, all users and groups are still allowed access.  There’s absolutely no mention of that in the finding what so ever.  Complete false sense of security, and utterly useless.  Just set AllowGroups in your sshd_config file and move on.  It does absolutely nothing for you what so ever, but you should have SSH and the Console turned off and in lockdown anyway.

Note: ESXi 5.0 is the last version that supported the concept of local groups.  As of ESXi 5.1 local group objects no longer exist.

GEN005524-ESXI5-000104 – SSH Daemon Must Not Permit GSSAPI, Unless Needed

This one is simple enough, mark GSSAPIAuthentication to No in the sshd_config file.  No is the default in openSSH if no configuration is given.

Note: We do not compile Kerberos support into the OpenSSH product shipped with ESXi.  This means any setting that is connected to Kerberos functionality (GSSAPI is one) may error in the log as an unknown setting.  This will not hurt your system.  You can comply to the finding by simply adding the setting and moving on, noting that you may get errors in your system log.  Never fear, it's not that you typed it wrong, it's that the functionality wasn't compiled in and the setting is useless.  Technically this should be PNF and removed from the STIG in later versions.

GEN005526-ESXI5-000105 – SSH Daemon Must Not Permit Kerberos, Unless Needed

Again, see above.  Simple enough to comply with the aforementioned catch, just mark KerberosAuthentication to No in the sshd_config file.

GEN005528-ESXI5-000106 – SSH Daemon Environment Variable Exclusions

So the finding is that you MUST NOT ACCEPT any, or ONLY ACCEPT environment variables pertaining to LOCALE.  So at a minimum we will accept locale, or we accept nothing, right?  Well the test and fix text doesn’t really support that fully.

So the default in openSSH is that unless specified explicitly we will accept no environment variables from the client.  Also true is if you specify AcceptEnv with no value in the sshd_config file.  That fits the rule description, right?  Yes, but lets look at the text closely.

If “AcceptEnv” is not set to  “LOCALE”  or the keyword/line is missing, this is a finding.

Wait a minute, if the keyword is missing it’s accepting nothing, and if the keyword by itself is present, it accepts nothing.  I know, we live in the “seeing is believing” IA world in DoD, but the description and the check/fix contradict one another.

Lets look again at the spirit of this rule.  It says must not accept any, or only accept locale as a client passed environment variable.  Looks to me like they are actually saying you must accept the locale environment variable and have no other choice.  The option to accept no client environment variables isn’t possible with the way check/fix is written.  What it should say is “If “AcceptEnv” is not set to “LOCALE”, set to nothing, or the keyword/line is missing, this is a finding”.  Yes, I know, if the keyword is missing it accepts nothing, but I’m bowing to the seeing is believing culture here.

GEN005530-ESXI5-000107 – SSH Daemon Must Not Permit User Environment Settings

Pretty straight forward.  The default in openSSH is no, but you should set it via sshd_config and comply.  Nice and easy for once.

GEN005531-ESXI5-000108 – SSH Daemon Must Not Permit Tunnels

Like a previous finding this is a bit incomplete because PermitTunnel is not a boolean type keyword.  There are four (4) possible values for PermitTunnel in openSSH {yes|point-to-point|ethernet|no}.  So, in a world of specifics you can once again comply with this finding by setting it to {point-to-point|ethernet} and not meet the spirit of its intentions.  All they have you check for is if the keyword exists and it says “yes”.  Just put it into your sshd_config file and set it to no.  That’s what they meant, albeit not expressed appropriately.

GEN005536-ESXI5-000110 – SSH Daemon Strict Mode Checking

This one is straight forward, although not really needed in an ESXi environment.  Just set StrictModes to yes.  By the way, it’s yes by default in openSSH.

GEN005537-ESXI-000111 – SSH Daemon Must Use Privilege Separation

Can you tell they did not test this prior to releasing it?

This one is really tricky.  In ESXi 5.0 you cannot make this work, period, because you need chroot and that is not compiled into 5.0.  So if you’re running this against ESXi 5.0 you’re out of luck, it’s open.  Now, in 5.1 and 5.5 you can configure this just fine, with a little finesse.  Check out how to do it here Blog Series: ESXi 5 STIG – ESXi Server SSHD Privilege Separation

GEN005538-ESXI5-000112 – SSH Daemon Must Not Allow rhosts RSA Authentication

Rhosts RSA Authentication is a Protocol v1 option ONLY.  It doesn’t even apply for Protocol 2.  If we’re enforcing Protocol 2, why is this a CAT II finding if not set?  You got me.  Just set it per the fix text and move on.

GEN005539-ESXI5-000113 – SSH Daemon Must Not Allow Compression

Hey, they finally got a multiple option check text correct.  The options available are {yes|delayed|no} and they called them all out, but wait… the description says the following.

The SSH daemon must not allow compression or must only allow compression after successful authentication.

So we can either setup no compression, or use compression after successful authentication.  Does the check or fix text support this choice?  LOL, nope!  Once again the check itself offers two choices but the check/fix forces you into one.  How’s that?  Well guess what the delayed value does.  That’s right, it delays compression until after authentication.  What the check should have read, if the title is the true intent here, is if it’s set to yes or not present.  Delayed is a proper setting, if you go by the description that is.  If you go by the seeing-is-beliving principle you are better off just setting this to no, even though delayed should be allowed here.  By the way, delayed is the default if nothing is configured

Ok, that’s everything specifically for SSH Daemon.  Let’s review.  The following is a list of STIG IDs that if you followed this post you are in compliance with, sans the obvious issues pointed out above.

In Compliance

  • SRG-OS-000027-ESXI5
  • SRG-OS-000033-ESXI5
  • SRG-OS-000109-ESXI5
  • SRG-OS-000112-ESXI5
  • SRG-OS-000250-ESXI5
  • GEN005515-ESXI5-000100
  • GEN005517-ESXI5-000101
  • GEN005519-ESXI5-000102
  • GEN005521-ESXI5-000103
  • GEN005524-ESXI5-000104
  • GEN005526-ESXI5-000105
  • GEN005528-ESXI5-000106
  • GEN005530-ESXI5-000107
  • GEN005531-ESXI5-000108
  • GEN005536-ESXI5-000110
  • GEN005538-ESXI5-000112
  • GEN005539-ESXI5-000113

Now, the following is a “sort of” and depends on your IA group.  I know there isn’t a such thing as “sort of” security compliance in DoD IA terms, but I wanted to draw a distinction between the one that technically is compliant, just written up incorrectly by DISA, and the one that cannot be put into compliance at all.

Sort-Of Compliance

  • SRG-OS-000023-ESXI5 (Banner if /etc/issue, fully compliant if you build /etc/banner at boot)

Non-Compliant in Version 5.0, Can Be Compliant in 5.1 and Above

  • GEN005537-ESXI5-000111 (Privilege Separation)

That’s it, SSH Daemon settings are complete for now.  There are a couple of additional settings that will go in here that are covered in later articles, but this is the bulk of it.

Feel free to comment below, provide insight, corrections, or suggestions.  Next up, SSH Client settings.

21 thoughts on “Blog Series: ESXi 5 STIG – ESXi Server SSH Daemon

  1. Eric thanks for sharing the insight. I wish we had more information like this for all the STIGs. My question is on does @wlam scripts help fix the STIG findings?

  2. Hi Luis, thanks for the feedback! My hope is to keep this going for the entire ESXi STIG. This serves as my scratch-pad for comments toward the next revision by DISA as well. Hopefully I’ll get another part out this week while at VMworld.

    As for William’s scripts I’m uncertain at the moment. On my todo list is a correlation between the STIG and the Hardening guide. Something that to my knowledge doesn’t exist at the moment. Once we have that we can overlay William’s scripts and fill the gaps.

  3. Eric,
    Thanks for the great article. My very first question when I contacted support about the STIG was “do you have a knowledge base article that addresses the STIG”. I only learned about your blog a month later and after two series of escalations after many e-mails to Level 1 support. Anyway…

    SRG-OS-000023-ESXI5 – SSH Daemon Logon Banner
    -My solution was to locate the file on persistent storage [MyDataStore]/Banner and point to it in the sshd_config file. Easier than creating a persistent /etc/issue file.

    SRG-OS-000109-ESXI5 – No Root Logon via SSH
    “Just make sure you have access to the DCUI console before setting this, if you’re working over SSH right now.”
    -I wouldn’t have got the joke if I hadn’t tested this. What you meant to say is that although 5.1 implements named user logins for DCUI, it doesn’t for SSH. Once you change this setting, you have effectively disabled SSH since only root can log into SSH. I think you have to leave this as an open if you want SSH. Note there is another check that has you disable SSH and you have to manually enable it when you need it. I also noted that the SSH timout check changes the service to disabled after 900 seconds (the STIG is assumes minutes so it’s wrong on that one too). I suspect the disable after 900 seconds isn’t what the STIG meant to do anyway; it really should just log you out after 900 seconds of inactivity. I think you should include this one in your Not-Compliant list because although you could be technicially compliant by setting it, you have to unset it each time you want to use SSH and that seems to violate the spirit of the check. Also, can you put this in as a feature request, i.e. named user log-in for SSH (same as DCUI in 5.1). Can you confirm that no-named-user-login-for-SSH is still the same in 5.5?

    GEN005524-ESXI5-000104 – SSH Daemon Must Not Permit GSSAPI, Unless Needed
    -This generated an error in the log indicating that it wasn’t a supported option. Ideas?

    • Great feedback Albert, thank you. Taking the time to do this just makes the resource better for everyone. That was exactly what I wanted these to become, living references for everyone. As for the log taking some time to find I apologize. It is entirely unofficial; however I’m the one for VMware that manages this relationship with DISA so while not an official source I felt the need to document somewhere. As for a KB I doubt one will be created for a couple of reasons. We can talk offline Bout that.

      Login Banner – I would argue it’s six of one, half dozen another. Your method works perfectly, obviously, but /etc/issue persists on its own. Just edit and save, and possibly forcing the auto-backup. My comment to DISA was to change the STIG to reference issue vs. banner because the /etc/issue file is intended by us for that purpose. When I get a moment I may update the article to point out that issue is just one way to do it.

      No root login via SSH – Lol, agreed. I can word that better so my attempt at humor isn’t as obscure. Thanks for the feedback. I’ll also include how you can setup other SSH users other than root. As for the 900 seconds I’ve provided that as feedback to DISA but based on your comment perhaps I didn’t point it out. I’ll verify and update if necessary.

      GSSAPI – I’ll retest this. I don’t remember it causing an issue for me, but perhaps I missed it. I also need to verify we actually built this into our make for openSSH being used. It might be entirely NA and I’ll update DISA if so.

      Apologizes for being short. Foreign airport at the moment with limited access.

  4. Eric,
    I am still struggling with the SSH named-user log-in thing. You sort of seem to indicate it doesn’t work (but does with DCUI) in your article, but in your reply, you indicate there is some sort of work-around needed to get it working? Perhaps you could share? I did the UsePrivilegeSeparation fix and that didn’t have any effect.

    Thanks!
    Albert

    • Hi Albert,

      Sorry for the delay. Here’s the trick, it’s two fold and a little different since we removed vicfg from the console.

      * First off your ESXi Server must be joined to the domain. That’s an obvious one but worth noting.
      * Second, you must establish a permission (not propagated) in a VI client directly on the asset. IOW, if you want to give user albert@testdomain.com access that permission must be created in the client on the ESXi server.
      * Finally, on the console, edit the /etc/security/access.conf file for that user/group object removing the leading minus (-) and replacing it with a plus (+). This will enable SSH / console access for the user or group in question. Back this up with /sbin/auto-backup.sh. It will persist.

      Test – You should be able to SSH into the host using the AD backed account. You can use the NETBIOS or UPN style for your login (ie. ssh -l “TESTDOMAIN\albert”)

      • Eric,
        Thanks for the SSH named-user reply. I guess it’s not obvious to me why ESXi should be joined to the domain. Our current philosophy is that if your system gets compromised in-band, an attacker shouldn’t be able to use priviledge escalation to own the AD and then get access to the OOB. Even with an attacker inside your system you should still own the OOB and be able to start fixing things. Our OOB uses local-only authentication and any dual-homed box with OOB and in-band NICs is not allowed in AD (local-only authentication). Can we just put individual local users in /etc/security/access.conf?

        Thanks!
        Albert

        • Hey Albert,

          I completely agree with you on the rational. The trick here is the finding that states we cannot have any local users that are not AD backed. I’m on a plane at the moment so I cannot verify the category but it might be worthwhile to just justify the local user finding open and do it via the method you described. I certainly design and implement in that manner as well, well, except when I have to apply a STIG. 🙂

  5. Thanks for posting this — I wish I found it sooner. We are going through this at the moment and have the same gripes about UsePrivilegeSeparation, setting persistence, etc. I have had the same issues, but came up with some slightly different workarounds. In either case, it seems that the STIG for ESXi5 was not tested/vetted very well.

    Regarding SSH and GSSAPI – I had the same thing show up in my log files.

    2013-10-30T04:17:53Z sshd[12546]: reprocess config line 27: Unsupported option GSSAPIAuthentication
    2013-10-30T04:17:53Z sshd[12546]: reprocess config line 28: Unsupported option KerberosAuthentication

    I would say that it is indicative of them not being built this into the make for openSSH, as you mentioned.

    • Hi Dave, I’m glad the site was useful. Yes, I can confirm that Kerberos, and by extension GSSAPI, is NOT compiled in by us. I’ve been working with DISA to prove that out and remove the checks.

      Thank you for bringing it up though because I thought I updated the article with this information but it looks like I missed a spot.

  6. Like everyone else, I wished I had known about your site BEFORE starting to STIG my 5.0 environment (which is up to date on patches). Running through the latest version of the DISA STIG I’ve noticed a number of typos, duplications and format changes between rules which is quite frustrating — hope it gets better in the future.

    A problem that I’ve experienced is with GEN005520. This is the check for “ForwardX11” — not the “X11Forwarding” in rule GEN005519-ESXI5-000102 which doesn’t cause me a problem. If I include “ForwardX11 no” in my /etc/ssh/sshd_config file, I not longer gain access via Putty. The server immediately closes the connection.

    • Thanks for the good feedback. I’d also like to test your comment on X11 and see what I get. Might require a follow up article. Thanks!

      • Eric — I should have read through all of your blogs first before making my original comment. Ended up what I thought was a typo in the STIG was really the missing “ssh_config” file — forgot about the SSH Client config file. OpenSSH doesn’t like the SSH Client command “ForwardX11” in the SSH Daemon configuration file — basically shuts down SSH access. Realized that when read your File and Setting Persistence blog

  7. Hey Eric,

    Have you been able to implement sha2 successfully for sshd? When I try to use a sha2 variant hmac, it kills ssh and gives errors in auth.log.

    Any ideas?

    From /var/log/auth.log:
    2012-03-16T00:16:39Z sshd[802573]: fatal: /etc/ssh/sshd_config line 25: Bad SSH2 mac spec ‘hmac-sha2-256’.

    Excerpt from /etc/ssh/sshd_config:
    Ciphers aes128-ctr,aes192-ctr,aes256-ctr
    MACs hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96

    • I’m not even certain if VMware compiled in the SHA2 variants to the SSHD package in 5.x. I’m basing that on memory only of the build logs I reviewed a few months ago. Is it just that specific one that fails, or all SHA2 MACs?

      • All the SHA2 MACs fail. Probably right — I had a feeling SHA2 wasn’t compiled. I don’t know if it’s possible to check for that, to be honest.

        I asked VMware tech support but they didn’t know. Referred me to an IPSec document and closed the ticket ^^

        • There’s a way to tell, they just don’t know it. They have to look in the internal build system and review the logs of the automated build for that module. Best I remember SHA2 isn’t compiled in. I don’t have access anymore, nor is it in front of me, so I can’t be 100%, but I think that’s what I remember.

    • Hi Andrew. Love the script, I’ve gone through it and am impressed at the level of automation detail involved. If you have a DoD CAC Card you should see my post on this site regarding the Forge.mil VIB project. It’s something I tried to get started internally at VMware while I was there, but couldn’t get company buy-in. The above project is community driven to embed the necessary STIG settings into an easy to deploy VIB. Currently the only “easy” way to handle non-persistent files as well.

      What all offerings fail at thus far is a good way to allow the administrator using our scripts to pick and choose easily, assuming one has little to no knowledge of the shell or scripting. It’s a high hurdle to jump, but a necessary one a lot of time.

      If you get time, and have a CAC, the VIB project would value from your participation.

Comments are closed.