yaml paths Examples

1. Introduction
2. Scanning Data for Unprotected Secrets

Introduction

This page explores various real-world use-cases for the yaml-paths command-line tool.

Scanning Data for Unprotected Secrets

There are certainly better tools available for this sensitive topic. However, not all are free, open-source, or run as conveniently as the portable, light-weight yaml-paths tool. When using yaml-paths for this use-case, do so only for quick-scans -- something like an SCM trigger check -- and be sure to use another more sophisticated, purpose-built secrets detection tool on the back-end.

File: secrets.yaml

---
connections:
  databases:
    application:
      exposed-connection:
        host: exposed_host
        username: exposed_username
        password: exposed_password
        port: 5280
        schema: exposed_schema
      protected-connection:
        host: >
          ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw
          DQYJKoZIhvcNAQEBBQAEggEAXkr3X7jmdrMPSRbs1++RZOPww/63Ok3VhE6U
          kLYcvSecVwx+QsPScnBTVN4H6zTw+i24Zv9oN4lCBoXnvNE63RfF/F2Ty7kB
          QUOulHCabvnjgznxCi3F0X7/Nl5I/7qxTkgnwueg2NOE0YF5fptOquR3/3m9
          ri6M0VC43jV1ukUW5RWmTzM0j8Fznm6viAS36Y52AzVsvKt4/nEz3DLVt7iX
          szjU2rKLt6rWeA1iHL7NxQqKN9REUX1aBcONP1CuWs/zRLSG6/320JC7k4gZ
          pHsO+FzMedVscHrw1HPzckUWq76OjT0uIRv3SeaH1680vtwiAbXt9ZXqB1SV
          79UoJjA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBDiTDBx7ZQ2YW9qZaFm
          5asCgBBeJWEVJocFg+l7fBBiDtXQ]
        username: >
          ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw
          DQYJKoZIhvcNAQEBBQAEggEAyFLEfyxuJQMHoHNkksMmpyN4BNf1PlUBgHT8
          su4qqqStKSyhw2QA6sqL5JYfx1u1DPzkk5q9Hswf3eFosNyA+nqG/Q00ass7
          /z4KtYVcW2zCuBtqTmtOeWlhReB16sO1nrAprWxLDhEzJ+iIrSjeMnjY4+cn
          n+VV8etfmk8jenRDrqCikoIVPFw5YsGso+QZ4wF/GI5i7GmOGpuyplZ1GpgD
          djfa4xP86WyOTjmG58Q9zbo2R8QPRUb418e+EvAcrfWEvTSq7nLB6oo9ojuJ
          I27ISgbzuD/itPLHxsSWMHQc7xQ2MGA8RpXnjmISllMredS0rO0O17YwzMWy
          76P2zTBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBDlvWMk3AC7RSfBkMl7
          TGcUgCDMXuQOH/oSaVlSRPPbvFrylbcakqlB5rWMgjkZQb0rMA==]
        password: >
          ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw
          DQYJKoZIhvcNAQEBBQAEggEAFz0Wqg8Xzhc8ZcOdAKdf9gqUgJkM3hNj88KE
          zkI3w/yHYe2uQVrwTunu0HHSABOfiZERty3un1NUYeSKqG2pgFXrsij33H60
          Vw8KWeHWQ04UZ6UjmIL//gTrNWIderAMkQMfZas4izG/dtZHle+c3p7qep8S
          Ms1bDhXuHdzyVsdOl37e1wBOOQyenpJSyxSAknzbO3sxayOi6OgbwXWCjGeD
          XZOTdyxXA4w6CFW/wnUwr2WK59c/VaWQnEDV2w31OoOm5knLJR9t22c77kmc
          t4/766ae5JGboDYPR42FFzeizDaEY+9SjehkYtuLRpsPHmFPkczf4ypvS2Fh
          +0U5djBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAlQJRRU4/f2ti5rOr9
          EIAwgCBHCLcQzzeCku+v1wfisHXfrOVkh0+lqH6pOnPs+3/jcg==]
        port: >
          ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw
          DQYJKoZIhvcNAQEBBQAEggEAwEuR9Z/hoRRNhiv+AfanDCGPwYvWRiL82u5P
          3gdhyYh5sNIvZ47Dtcad7cx0SDJnUO6RqqX+yuQ0+Jr4+rKAb3vw/iYFEFRi
          GUqN3uK1BqMf/oO+G8z++06yeWSWm5icQc4GeNvEJBo5dR3onlOYYsgHCmp9
          KYNIQ2kHzWFzakl20Qp9WiqyUfJR2h7QCDEDdbGXGM+DJv84KTC9E33tV1Ug
          GvVY6gexsOSTxkeKOJhB1sYCSKgKXE2dexyMQqD8riZs4XMk9rntHMu1BY3q
          f5h7u3ytrc2tB3bWNXmWoO5bhH7RZR8yNyNTOcRJCDkn935PIsMrNG/kvk9X
          QQO1xTA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBjMZFexp5+0Wvr54gj
          fmyLgBBN2CIHLQVVBALS7SHZ1WcO]
        schema: >
          ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw
          DQYJKoZIhvcNAQEBBQAEggEANZeaGhjF5qXr/AKZu4QaK47GKxLLM1rctAWr
          sOU2teXb3uSHRUN/ldhVAuYXVs4Huxinl0XIqzS6zslTyhJUWPZZ0KDHbqP2
          udNoQnrQzr8Chis3ZcLh5PlyFdtWYhgDhOHUWwNer/YBcCFWooW0q62QHQMM
          oVDk0DTkF3mcaWBPSztmdlPUeatLa+G8XUUo15MmTFvvDV2ENR8oYjeYg3Oe
          tu/L4egAEyVN9tQf+7rDwpRGJzhvvcj6wbs2bY9oa9RfAcl7bKDIHQvQ9N0B
          MO3b4ULmXd/JdCMcmUwUysfINj26K7z1QOa12q1kq/RyR6G2VjVFej8xdY93
          m05WZDA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAXUTLsChHpz0WN4m7I
          GvDjgBCPnNCiN7se8cVGDFn2zPql]

The sample data file demonstrates two database connection configurations: one that is fully exposed to anyone who can intercept or otherwise read the data and another which is protected via EYAML.

If all we wanted was a listing of every sensitive value, we could just search for everything of interest: yaml-paths --keynames --search='=~/(user)*_?name/' --search='=~pass_?(word|phrase)*' --search='=~/host_?(name)*/' --search='=~/schema_?(name)*/' --search='=port' secrets.yaml, which produces:

secrets.yaml/0[=~/(user)*_?name/]: connections.databases.application.exposed-connection.username
secrets.yaml/0[=~/(user)*_?name/]: connections.databases.application.protected-connection.username
secrets.yaml/0[=~pass_?(word|phrase)*]: connections.databases.application.exposed-connection.password
secrets.yaml/0[=~pass_?(word|phrase)*]: connections.databases.application.protected-connection.password
secrets.yaml/0[=~/host_?(name)*/]: connections.databases.application.exposed-connection.host
secrets.yaml/0[=~/host_?(name)*/]: connections.databases.application.protected-connection.host
secrets.yaml/0[=~/schema_?(name)*/]: connections.databases.application.exposed-connection.schema
secrets.yaml/0[=~/schema_?(name)*/]: connections.databases.application.protected-connection.schema
secrets.yaml/0[=port]: connections.databases.application.exposed-connection.port
secrets.yaml/0[=port]: connections.databases.application.protected-connection.port

However, for our use-case, we are interested only in unprotected secrets. For this, we except (exclude) encrypted values, like so: yaml-paths --keynames --search='=~/(user)*_?name/' --search='=~pass_?(word|phrase)*' --search='=~/host_?(name)*/' --search='=~/schema_?(name)*/' --search='=port' --except='^ENC[' secrets.yaml, which reduces the matches to just:

secrets.yaml/0[=~/(user)*_?name/]: connections.databases.application.exposed-connection.username
secrets.yaml/0[=~pass_?(word|phrase)*]: connections.databases.application.exposed-connection.password
secrets.yaml/0[=~/host_?(name)*/]: connections.databases.application.exposed-connection.host
secrets.yaml/0[=~/schema_?(name)*/]: connections.databases.application.exposed-connection.schema
secrets.yaml/0[=port]: connections.databases.application.exposed-connection.port

The results display every exposed secret's YAML Path, which file each was found in (secrets.yaml), which sub-document (/0), and which search term matched. You can further tailor the output to -- for example -- eliminate any of those components of the results or even expose the value at the end of each matching YAML Path.

Note that this is a naive scan for secrets. While you can certainly add more --search (to capture more variations of these names) and --except expressions (to curtail false-positives), determined users can always craft bogus keys which would elude detection, anyway. As such, this tool should be used only where instant results are desired while the data is rigorously scrutinized by other special-purpose tools afterward.