Add a new AutoDiscovery Operator for automated Scans in Kubernetes Clusters by J12934 · Pull Request #461 · secureCodeBox/secureCodeBox

J12934 · 2021-05-31T20:47:00Z

Description

This PR adds a "AutoDiscovery" feature for workloads running inside kubernetes.
Currently only supports http / https types of workloads by "auto-discovering" kubernetes services and starting scans against them.

For more info check out the readme of the AutoDiscovery subfolder: https://github.com/secureCodeBox/secureCodeBox/tree/auto-discovery/auto-discovery/kubernetes

Todos

License headers
Resources limits configuration via helm chart

nigthknight

Great Work!
I just have some questions and minor issues.

Why do we need a kubernetes directory?
I think it would be best to include the short SPDX Header instead of the Apache Header
Trying to run the tests locally with make test fails with

•STEP: tearing down the test environment

------------------------------
Failure [20.014 seconds]
[AfterSuite] AfterSuite
/home/yfuhrmeister/Projects/scb/secureCodeBox/auto-discovery/kubernetes/controllers/suite_test.go:119

  Unexpected error:
      <errors.aggregate | len:1, cap:1>: [
          {
              s: "timeout waiting for process kube-apiserver to stop",
          },
      ]
      timeout waiting for process kube-apiserver to stop
  occurred

  /home/yfuhrmeister/Projects/scb/secureCodeBox/auto-discovery/kubernetes/controllers/suite_test.go:122

nigthknight · 2021-06-01T06:14:01Z

auto-discovery/kubernetes/Dockerfile

+COPY pkg/ pkg/
+
+# Build
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go


Is there a reason not to use the Makefile at this point as well?

This is directly taken from the defaukt kubebuilder starting point, so not sure what the exact reasoning here is to doing it like this, I'd guess that this gives them more controll over the flags used for the go build but I'm not sure.

auto-discovery/kubernetes/api/v1/autodiscoveryconfig_types.go

auto-discovery/kubernetes/api/v1/groupversion_info.go

auto-discovery/kubernetes/api/v1/zz_generated.deepcopy.go

auto-discovery/kubernetes/main.go

auto-discovery/kubernetes/pkg/util/annotations.go

auto-discovery/kubernetes/pkg/util/annotations_test.go

.github/workflows/ci.yaml

Co-authored-by: Yannik Fuhrmeister <12710254+fuhrmeistery@users.noreply.github.com>

J12934 · 2021-06-01T13:48:15Z

Great Work!
I just have some questions and minor issues.

1. Why do we need a `kubernetes` directory?

2. I think it would be best to include the short SPDX Header instead of the Apache Header

3. Trying to run the tests locally with `make test` fails with

•STEP: tearing down the test environment

------------------------------
Failure [20.014 seconds]
[AfterSuite] AfterSuite
/home/yfuhrmeister/Projects/scb/secureCodeBox/auto-discovery/kubernetes/controllers/suite_test.go:119

  Unexpected error:
      <errors.aggregate | len:1, cap:1>: [
          {
              s: "timeout waiting for process kube-apiserver to stop",
          },
      ]
      timeout waiting for process kube-apiserver to stop
  occurred

  /home/yfuhrmeister/Projects/scb/secureCodeBox/auto-discovery/kubernetes/controllers/suite_test.go:122

@fuhrmeistery

The kubernetes directory is so that we can add additional "auto-discoveries" for different cloud / runtime environments in the future, not just kubernetes. This is mentioned in the readme in the auto-discovery dir.
Yup, good point the existing headers are automatically setup by kubebuilder didn't setup the spdx header up yet...
Not sure whats going on there, seems to work okay for my machine and the CI, looks like this is only failing to tear down the api server, is your linux distro configured in a unusual way which might interfer with that?

nigthknight · 2021-06-01T14:20:52Z

@fuhrmeistery

1. The kubernetes directory is so that we can add additional "auto-discoveries" for different cloud / runtime environments in the future, not just kubernetes. This is mentioned in the readme in the `auto-discovery` dir.

2. Yup, good point the existing headers are automatically setup by kubebuilder didn't setup the spdx header up yet...

3. Not sure whats going on there, seems to work okay for my machine and the CI, looks like this is only failing to tear down the api server, is your linux distro configured in a unusual way which might interfer with that?

Yes, I think you are right. Tested on another machine and everything was fine o.O

J12934 · 2021-06-01T15:11:37Z

Mh interesting, hopefully that's actually related to your system and not that the test suite is flaky 😅

Signed-off-by: GitHub Actions <securecodebox@iteratec.com>

J12934 · 2021-07-09T13:06:41Z

I fixed some typos in the helm-docs templates this has exploded as this has changed the doc files in every component. PR should still be reviewable by just checking the changes auto-discovery/ and .helm-docs/ folder.
I changed the helm-docs action to check the checksum of the downloaded binary and mv'ed it to the /usr/local/bin so that we don't have to juggle with the correct relative path every time

This should make the PR look a bit more green :) Signed-off-by: Jannik Hollenbach <jannik.hollenbach@iteratec.com>

nigthknight

Looks great to me 👍

J12934 added 12 commits May 31, 2021 13:29

Add first stable(ish) kubernetes AutoDiscovery version to the main repo

0d5c8ce

Add ci for auto-discovery

34f74fc

Change job name to better fit in to the parent name prefix

bbde1d6

Remove temporary workaround to install crds from local folder

477eb7a

Update go pkg paths

ada57c5

Update JuiceShop example

bec1356

Fix default scan config in auto-discovery helm config

f013c0d

Update docs

2bd5f0b

Update docs

1830393

Update chart meta infos

3321b99

Remove unused cloud build config

018c8b8

Reorder install instructions

bb3b352

J12934 added enhancement New feature or request auto-discovery labels May 31, 2021

J12934 self-assigned this May 31, 2021

Fix resource inclusion mode docs

6a45988

nigthknight reviewed Jun 1, 2021

View reviewed changes

Use working-dir attribute instead of cd'ing into the folder

629e321

Co-authored-by: Yannik Fuhrmeister <12710254+fuhrmeistery@users.noreply.github.com>

rfelber added this to the v2.8.0 milestone Jun 3, 2021

rfelber linked an issue Jun 3, 2021 that may be closed by this pull request

Implement a Scan Autodiscovery for K8S Services #361

Closed

rfelber and others added 6 commits June 3, 2021 11:33

Merge branch 'main' into auto-discovery

7a530ba

Move default image used to docker hub

b795dee

Acutally use mounted config file in auto-discovery

8245dc5

Fix config

7b4c125

Move to v3.0.0-beta1

d441d97

Generate helm-docs

83fc931

rfelber removed this from the v2.8.0 milestone Jun 4, 2021

J12934 and others added 17 commits July 9, 2021 11:24

Clear up comment

ba4dcb8

Add license headers

533cf5d

Fix typo in operator docs

a2888a5

tmp

a8d13d5

Merge branch 'main' into auto-discovery

d54994d

Regenerate readmes after typo fixes

7038879

Split core helm-doc jobs

d3f7d14

Verify checksum of helm-docs installation

865fad4

Move helm-docs to /usr/local/bin

b0dc489

Move with sudo

1718513

Use helm-docs from user path

f8db10c

Updating Helm Docs

3dc7d63

Signed-off-by: GitHub Actions <securecodebox@iteratec.com>

Remove the word scanType from deployment section

49d815d

Updating Helm Docs

a658d4a

Signed-off-by: GitHub Actions <securecodebox@iteratec.com>

Expose pod resources and securityContext settings via helm

c3083c3

Updating Helm Docs

e5c8cd9

Signed-off-by: GitHub Actions <securecodebox@iteratec.com>

Add small note on the checksum

4cc8630

J12934 marked this pull request as ready for review July 9, 2021 12:58

J12934 requested a review from nigthknight July 9, 2021 12:58

J12934 and others added 4 commits July 9, 2021 15:02

Move helm-docs to /usr/local/bin instead of /usr/bin

de6f136

Include Development section in new readme

f63235a

Remove old REAME gotmpl

0db3ad1

Updating Helm Docs

bfdfc3d

Signed-off-by: GitHub Actions <securecodebox@iteratec.com>

Trigger CI

808998d

This should make the PR look a bit more green :) Signed-off-by: Jannik Hollenbach <jannik.hollenbach@iteratec.com>

nigthknight approved these changes Jul 12, 2021

View reviewed changes

Merge branch 'main' into auto-discovery

cd82b68

J12934 merged commit 92bae12 into main Jul 12, 2021

J12934 deleted the auto-discovery branch July 12, 2021 08:59

rfelber changed the title ~~Add AutoDiscovery for Kubernetes~~ Add a new AutoDiscovery Operator for automated Scans in Kubernetes Clusters Aug 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a new AutoDiscovery Operator for automated Scans in Kubernetes Clusters#461

Add a new AutoDiscovery Operator for automated Scans in Kubernetes Clusters#461
J12934 merged 48 commits intomainfrom
auto-discovery

J12934 commented May 31, 2021 •

edited

Loading

Uh oh!

nigthknight left a comment

Uh oh!

nigthknight Jun 1, 2021

Uh oh!

J12934 Jun 1, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

J12934 commented Jun 1, 2021 •

edited

Loading

Uh oh!

nigthknight commented Jun 1, 2021

Uh oh!

J12934 commented Jun 1, 2021

Uh oh!

J12934 commented Jul 9, 2021

Uh oh!

nigthknight left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

J12934 commented May 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Todos

Uh oh!

nigthknight left a comment

Choose a reason for hiding this comment

Uh oh!

nigthknight Jun 1, 2021

Choose a reason for hiding this comment

Uh oh!

J12934 Jun 1, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

J12934 commented Jun 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nigthknight commented Jun 1, 2021

Uh oh!

J12934 commented Jun 1, 2021

Uh oh!

J12934 commented Jul 9, 2021

Uh oh!

nigthknight left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

J12934 commented May 31, 2021 •

edited

Loading

J12934 commented Jun 1, 2021 •

edited

Loading