New Feature:

- (#137) RASP: add noSQL Injection protection support for the Go MongoDB driver
  `go.mongodb.org/mongo-driver/mongo`. This protection can be configured at
  <https://my.sqreen.com/application/goto/modules/rasp/details/nosql_injection>.

Internal Changes:

- (#138) Health-check the HTTPS connectivity to the new backend API
  `ingestion.sqreen.com` before using it. Fallback to the usual
  `back.sqreen.com` in case of a connection issue. Therefore, the agent can take
  up to 30 seconds to connect to Sqreen if the health-check timeouts. Please
  make sure to add this new  firewall and proxy configurations.

- (#136) Add support to attach multiple security protections per hook point.

Fixes:

- (#140) Fix the In-App WAF metadata PII scrubbing to also match substrings.

Author: Julio Guerra

CHANGELOG.md

@@ -1,4 +1,27 @@
-# v0.12.1
+# v0.13.0 - 24 July 2020
+
+## New Feature
+
+- (#137) RASP: add noSQL Injection protection support for the Go MongoDB driver
+  `go.mongodb.org/mongo-driver/mongo`. This protection can be configured at
+  <https://my.sqreen.com/application/goto/modules/rasp/details/nosql_injection>.
+
+## Internal Changes
+
+- (#138) Health-check the HTTPS connectivity to the new backend API
+  `ingestion.sqreen.com` before using it. Fallback to the usual
+  `back.sqreen.com` in case of a connection issue. Therefore, the agent can take
+  up to 30 seconds to connect to Sqreen if the health-check timeouts. Please
+  make sure to add this new  firewall and proxy configurations. 
+
+- (#136) Add support to attach multiple security protections per hook point.
+
+## Fixes
+
+- (#140) Fix the In-App WAF metadata PII scrubbing to also match substrings.
+
+
+# v0.12.1 - 13 July 2020
 
 ## Fixes
 
@@ -19,7 +42,7 @@
 - (eeb1dca) Avoid copying the metadata returned by the In-App WAF.
 
 
-# v0.12.0
+# v0.12.0 - 6 July 2020
 
 ## New Features
 
@@ -53,7 +76,7 @@
 - (794d6e2) Allow port numbers in the `X-Forwarded-For` header.
 
 
-# v0.11.0
+# v0.11.0 - 19 June 2020
 
 ## New Features
 
@@ -90,14 +113,14 @@
 - (#114) Add Goroutine Local Storage (GLS) support through static instrumentation of the Go runtime.
 
 
-# v0.10.1
+# v0.10.1 - 5 June 2020
 
 ## Fix
 
 - (#116) Fix the instrumentation tool ignoring vendored packages, leading to
   missing hook points in the agent.
 
-# v0.10.0
+# v0.10.0 - 20 May 2020
 
 ## New Features
 
@@ -136,7 +159,7 @@
 
 - Document PII scrubbing configuration at <https://docs.sqreen.com/go/configuration/#personally-identifiable-information-scrubbing>.
 
-# v0.9.1
+# v0.9.1 - 31 March 2020
 
 ## Fixes
 
@@ -150,7 +173,7 @@
 - (#101) Prevent starting the agent when the instrumentation tool and agent
   versions are not the same.
 
-# v0.9.0
+# v0.9.0 - 19 February 2020
 
 This new major version says farewell to the `beta` and adds SQL-injection
 run time protection thanks the first building blocks of [RASP][RASP-Wikipedia]
@@ -233,7 +256,7 @@ Because we now want a stable public API, find below the breaking changes:
     compiled as a Go module. This is also shown by the dashboard when the list
     of dependencies is empty.
 
-# v0.1.0-beta.10
+# v0.1.0-beta.10 - 24 January 2020
 
 ## Breaking Change
 
@@ -264,7 +287,7 @@ Because we now want a stable public API, find below the breaking changes:
 - (#92) Vendoring using `go mod vendor` could lead to compilation errors due to
   missing files.
 
-# v0.1.0-beta.9
+# v0.1.0-beta.9 - 19 December 2019
 
 ## New Features
 
@@ -283,7 +306,7 @@ Because we now want a stable public API, find below the breaking changes:
 - The In-App WAF has been intensively optimized so that large requests can no longer impact
   its execution time. (#83)
 
-# v0.1.0-beta.8
+# v0.1.0-beta.8 - 15 October 2019
 
 ## Internal Changes
 
@@ -292,7 +315,7 @@ Because we now want a stable public API, find below the breaking changes:
   - Ignore WAF timeout errors and add more context when reporting an error (#80).
   - Update the libsqreen to v0.4.0 to add support for the `@pm` operator.
 
-# v0.1.0-beta.7
+# v0.1.0-beta.7 - 26 September 2019
 
 ## Breaking Changes
 
@@ -319,7 +342,7 @@ Because we now want a stable public API, find below the breaking changes:
 - Fix a compilation error on 32-bit target architectures.
 
 
-# v0.1.0-beta.6
+# v0.1.0-beta.6 - 25 July 2019
 
 ## New Features
 
@@ -354,7 +377,7 @@ Because we now want a stable public API, find below the breaking changes:
   log-level.
 
 
-# v0.1.0-beta.5
+# v0.1.0-beta.5 - 23 May 2019
 
 ## New Features
 
@@ -380,7 +403,7 @@ Because we now want a stable public API, find below the breaking changes:
     processing loop.
 
 
-# v0.1.0-beta.4
+# v0.1.0-beta.4 - 16 April 2019
 
 This release adds the ability to block IP addresses or users into your Go web
 services by adding support for [Security Automation] according to your
@@ -440,7 +463,7 @@ Note that redirecting users or IP addresses is not supported yet.
 - Avoid performing multiple times commands within the same command batch. (51)
 
 
-# v0.1.0-beta.3
+# v0.1.0-beta.3 - 22 March 2019
 
 ## New Features
 
@@ -477,15 +500,15 @@ Note that redirecting users or IP addresses is not supported yet.
   self-managing the initializations. (#28)
 
 
-# v0.1.0-beta.2
+# v0.1.0-beta.2 - 14 February 2019
 
 ## New feature
 
 - Add a new `Identify()` method allowing to explicitly associate a user to the
 current request. As soon as we add the support for the security reponses, it
 will allow to block users (#26).
 
-# v0.1.0-beta.1
+# v0.1.0-beta.1 - 7 February 2019
 
 This version is a new major version towards the v0.1.0 as it proposes a new and
 stable SDK API, that now will only be updated upon user feedback. So please,

README.md

@@ -8,7 +8,6 @@
 [![GoDoc](https://godoc.org/github.com/sqreen/go-agent?status.svg)](https://godoc.org/github.com/sqreen/go-agent)
 [![Go Report Card](https://goreportcard.com/badge/github.com/sqreen/go-agent)](https://goreportcard.com/report/github.com/sqreen/go-agent)
 [![Build Status](https://dev.azure.com/sqreenci/Go%20Agent/_apis/build/status/sqreen.go-agent?branchName=master)](https://dev.azure.com/sqreenci/Go%20Agent/_build/latest?definitionId=8&branchName=master)
-[![Sourcegraph](https://sourcegraph.com/github.com/sqreen/go-agent/-/badge.svg)](https://sourcegraph.com/github.com/sqreen/go-agent?badge)
 
 After performance monitoring (APM), error and log monitoring it’s time to add a
 security component into your app. Sqreen’s microagent automatically monitors

go.mod

@@ -29,8 +29,8 @@ require (
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/spf13/viper v1.3.2
 	github.com/sqreen/go-libsqreen v0.7.0
-	github.com/sqreen/go-sdk/signal v1.0.0
-	github.com/stretchr/testify v1.5.1
+	github.com/sqreen/go-sdk/signal v1.1.0
+	github.com/stretchr/testify v1.6.1
 	golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 // indirect
 	golang.org/x/net v0.0.0-20200513185701-a91f0712d120
 	golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect
@@ -40,4 +40,5 @@ require (
 	gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
 	gopkg.in/go-playground/assert.v1 v1.2.1
 	gopkg.in/go-playground/validator.v8 v8.18.2 // indirect
+	gopkg.in/yaml.v2 v2.3.0 // indirect
 )

go.sum

@@ -101,12 +101,10 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/spf13/viper v1.3.2 h1:VUFqw5KcqRf7i70GOzW7N+Q7+gxVBkSSqiXB12+JQ4M=
 github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
-github.com/sqreen/go-libsqreen v0.6.1 h1:+SHH3h8qHhINEzgRVqTZ40YxqwDjSVxU5r4isUeg+C8=
-github.com/sqreen/go-libsqreen v0.6.1/go.mod h1:D324eoKlZGfW+TF3WGg+2fUtpdrI+cEK5UYwpxfaeUc=
 github.com/sqreen/go-libsqreen v0.7.0 h1:MRX/KB5lX3O6ucvmTUap6iSDt27bM+76MQpuDNjL+1o=
 github.com/sqreen/go-libsqreen v0.7.0/go.mod h1:D324eoKlZGfW+TF3WGg+2fUtpdrI+cEK5UYwpxfaeUc=
-github.com/sqreen/go-sdk/signal v1.0.0 h1:WNjufvcjKYOgSZHPCwqG0Od5eVAD8wxwmiIe6ZCqoNE=
-github.com/sqreen/go-sdk/signal v1.0.0/go.mod h1:UksuO4mxxDMFw3el+R9mW9tmCgdc94WiDcGuCXU/pwU=
+github.com/sqreen/go-sdk/signal v1.1.0 h1:l22lqlUNDlEaqsNjpgVelGteBCwGodZqUDPUMBOLzhE=
+github.com/sqreen/go-sdk/signal v1.1.0/go.mod h1:XWJV0TzuoN6PotzRn4YSe6fhTxyw67yRpVYr9NJTzto=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48=
 github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
@@ -115,8 +113,8 @@ github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
-github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
-github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
+github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
+github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 h1:3SVOIvH7Ae1KRYyQWRjXWJEA9sS/c/pjvH++55Gr648=
 github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
 github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
@@ -190,3 +188,6 @@ gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
 gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 h1:tQIYjPdBoyREyB9XMu+nnTclpTYkz2zFM+lzLJFO4gQ=
+gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

internal/adapter.go

@@ -334,6 +334,15 @@ func newMetricsAPIAdapter(logger plog.ErrorLogger, expiredMetrics map[string]*me
 	return metricsArray
 }
 
+type variousInfoAPIAdapter struct {
+	*appInfoAPIAdapter
+	sqreenDomains api.SqreenDomainStatusMap
+}
+
+func (v variousInfoAPIAdapter) GetSqreenDomains() api.SqreenDomainStatusMap {
+	return v.sqreenDomains
+}
+
 type appInfoAPIAdapter app.Info
 
 func (a *appInfoAPIAdapter) unwrap() *app.Info { return (*app.Info)(a) }

internal/agent.go

@@ -228,9 +228,9 @@ func New(cfg *config.Config) *AgentType {
 		logger.Info(message)
 		return nil
 	}
-	// TODO: agent.Health() + waf.Health()
+
 	if waf.Version() == nil {
-		message := fmt.Sprintf("in-app waf disabled: cgo was disabled during the program compilation while required by the in-app waf")
+		message := "in-app waf disabled: cgo was disabled during the program compilation while required by the in-app waf"
 		backend.SendAgentMessage(logger, cfg, message)
 		logger.Info("agent: ", message)
 	}
@@ -240,9 +240,11 @@ func New(cfg *config.Config) *AgentType {
 	sdkMetricsPeriod := time.Duration(cfg.SDKMetricsPeriod()) * time.Second
 	logger.Debugf("agent: using sdk metrics store time period of %s", sdkMetricsPeriod)
 
-	piiScrubber, err := sqsanitize.NewScrubber(cfg.StripSensitiveKeyRegexp(), cfg.StripSensitiveValueRegexp(), config.ScrubberRedactedString)
+	piiScrubber := sqsanitize.NewScrubber(cfg.StripSensitiveKeyRegexp(), cfg.StripSensitiveValueRegexp(), config.ScrubberRedactedString)
+
+	client, err := backend.NewClient(cfg.BackendHTTPAPIBaseURL(), cfg.BackendHTTPAPIProxy(), logger)
 	if err != nil {
-		logger.Error(sqerrors.Wrap(err, "ecdsa public key"))
+		logger.Error(sqerrors.Wrap(err, "agent: could not create the backend client"))
 		return nil
 	}
 
@@ -264,7 +266,7 @@ func New(cfg *config.Config) *AgentType {
 		cancel:      cancel,
 		config:      cfg,
 		appInfo:     app.NewInfo(logger),
-		client:      backend.NewClient(cfg.BackendHTTPAPIBaseURL(), cfg.BackendHTTPAPIProxy(), logger),
+		client:      client,
 		actors:      actor.NewStore(logger),
 		rules:       rulesEngine,
 		piiScrubber: piiScrubber,
@@ -341,7 +343,7 @@ func (a *AgentType) Serve() error {
 
 	token := a.config.BackendHTTPAPIToken()
 	appName := a.config.AppName()
-	appLoginRes, err := appLogin(a.ctx, a.logger, a.client, token, appName, a.appInfo, a.config.UseSignalBackend())
+	appLoginRes, err := appLogin(a.ctx, a.logger, a.client, token, appName, a.appInfo, a.config.DisableSignalBackend())
 	if err != nil {
 		if xerrors.Is(err, context.Canceled) {
 			a.logger.Debug(err)
@@ -586,11 +588,14 @@ func stopTimer(t *time.Timer) {
 
 func (m *eventManager) Loop(ctx context.Context, client *backend.Client) {
 	var (
-		stalenessTimer = time.NewTimer(m.maxStaleness)
+		// We can't create a stopped timer so we initializae it with a large value
+		// of 24 hours and stop it immediately. Calls to Reset() will correctly
+		// set the configured timer value.
+		stalenessTimer = time.NewTimer(24 * time.Hour)
 		stalenessChan  <-chan time.Time
 	)
-	defer stopTimer(stalenessTimer)
 	stopTimer(stalenessTimer)
+	defer stopTimer(stalenessTimer)
 
 	batch := make([]Event, 0, m.count)
 	for {
@@ -647,7 +652,7 @@ func (m *eventManager) sendBatch(ctx context.Context, client *backend.Client, ba
 		if _, err := m.agent.piiScrubber.Scrub(event, nil); err != nil {
 			// Only log this unexpected error and keep the event that may have been
 			// partially scrubbed.
-			m.agent.logger.Error(errors.Wrap(err, "could not send the event batch"))
+			m.agent.logger.Error(errors.Wrap(err, "could not scrub the event"))
 		}
 		req.Batch = append(req.Batch, *api.NewBatchRequest_EventFromFace(event))
 	}