Friday, May 14, 2021

OWASP test in Release Pipeline

 In this page we are going to add some tasks in Azure Release pipeline to do the tests.

Prerequisites

There is already a docker image containing  Zap2 files and a Python file called zap-baseline.py to run the process. The image is called owasp/zap2docker-stable requires a shared folder to put the report in it. To mount a file share I use a storage account in azure containing the shared location called security. I generate the Key to access the shared location and start the process.

When process has been completed you need to have a file called OWASPToNUnit3.xslt to convert the report into an NUnit file that can be published as a test result.

OWASP Stage Tasks

 There are 3 tasks in this stage:

  1. OWASP in Azure CLI 
    which stes up a Container Instance that runs the tests

  2.  Transforming PowerShell Script
    which uses a PowerShell script to transform the result into NUnit

  3. Publish Test Results
    which makes the result visible in the pipeline as Test Results

Stage Tasks Yaml

steps: - task: AzureCLI@2 displayName: 'OWASP in Azure CLI ' inputs: azureSubscription: 'Owasp_grp_sp' scriptType: ps scriptLocation: inlineScript inlineScript: | $key='"the-Key-to-Storage-Account-shared-location=="' $ZAP_COMMAND="/zap/zap-baseline.py -t """"https://the-url-to-test.something"""" -x OWASP-ZAP-Report.xml" az container create `
--resource-group owasp_grp `
--name owasp ` --image owasp/zap2docker-stable ` --ip-address public `
--ports 8080 `
--azure-file-volume-account-name owaspstore1000 `
--azure-file-volume-account-key $key `
--azure-file-volume-share-name security `
--azure-file-volume-mount-path /zap/wrk/ `
--command-line $ZAP_COMMAND az storage file download `
--account-name owaspstore1000 `
--account-key $key `
-s security `
-p OWASP-ZAP-Report.xml `
--dest %SYSTEM_DEFAULTWORKINGDIRECTORY%\OWASP-ZAP-Report.xml
- powershell: | ## The powershell task for converting the test report $XslPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\_Managed-Security/OWASPToNUnit3.xslt" $XmlInputPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\OWASP-ZAP-Report.xml" $XmlOutputPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\Converted-OWASP-ZAP-Report.xml" $XslTransform = New-Object System.Xml.Xsl.XslCompiledTransform $XslTransform.Load($XslPath) $XslTransform.Transform($XmlInputPath, $XmlOutputPath) displayName: 'Transforming PowerShell Script'

- task: PublishTestResults@2 displayName: 'Publish Test Results Converted-OWASP-ZAP-Report.xml' inputs: testResultsFormat: NUnit testResultsFiles: 'Converted-OWASP-ZAP-Report.xml'

Wednesday, May 12, 2021

Docker and Webapp

 In this page we will create a WebApp that serves a Docker image and see how to put it in CI/CD.

Create a Web App

If you are creating a web app in Azure portal, start with selecting Docker Container as Publish property in Basics tab. Choose Linux as Operating System. For this example a Standard SKU would just do fine.
Next, on the Docker-tab select your existing azurecr.io container registry and specify the Image:Tag you want to serve.

My tempate looks something inline with the following:

"resources": [ { "apiVersion": "2018-11-01", "name": "[parameters('name')]", "type": "Microsoft.Web/sites", "location": "[parameters('location')]", "tags": {}, "dependsOn": [ "[concat('Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]" ], "properties": { "name": "[parameters('name')]", "siteConfig": { "appSettings": [ { "name": "DOCKER_REGISTRY_SERVER_URL", "value": "[parameters('dockerRegistryUrl')]" }, { "name": "DOCKER_REGISTRY_SERVER_USERNAME", "value": "[parameters('dockerRegistryUsername')]" }, { "name": "DOCKER_REGISTRY_SERVER_PASSWORD", "value": "[parameters('dockerRegistryPassword')]" }, { "name": "WEBSITES_ENABLE_APP_SERVICE_STORAGE", "value": "false" } ], "linuxFxVersion": "[parameters('linuxFxVersion')]", "appCommandLine": "", "alwaysOn": "[parameters('alwaysOn')]" }, "serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]", "clientAffinityEnabled": false } }, { "apiVersion": "2018-11-01", "name": "[parameters('hostingPlanName')]", "type": "Microsoft.Web/serverfarms", "location": "West Europe", "kind": "linux", "tags": {}, "dependsOn": [], "properties": { "perSiteScaling": false, "maximumElasticWorkerCount": 1, "isSpot": false, "reserved": true, "isXenon": false, "hyperV": false, "targetWorkerCount": 0, "targetWorkerSizeId": 0 }, "sku": { "Tier": "Standard", "Name": "S1", "size": "S1", "family": "S", "capacity": 1 } }

As a security matter I have to point out the fact that the Web App is connecting to Azure Container Registry using 3 configuration items i.e. Server Url, Server Username and Server Password. These items are visible in Azure Portal Configuration:
DOCKER_REGISTRY_SERVER_URL, DOCKER_REGISTRY_SERVER_USERNAME, DOCKER_REGISTRY_SERVER_PASSWORD

Deploy Docker Image

In both cases where the Docker image gets pulled from Container Registry, you need to restart the instance in Container Instance and also in Web App Docker instance.

An other option would be to move the pull task into Azure Pipeline using. My example is defined as follows:

steps: - task: AzureRmWebAppDeployment@4 displayName: 'Deploy Azure App Service' inputs: azureSubscription: '$(Parameters.ConnectedServiceName)' appType: 'Web App for Containers (Linux)' WebAppName: '$(Parameters.WebAppName)' DockerNamespace: 'https://securedcontainerregistry.azurecr.io' DockerRepository: 'https://securedcontainerregistry.azurecr.io/securedazurelib' DockerImageTag: 'latest' StartupCommand: ''


Monday, May 10, 2021

Docker and Container Registry in Azure

 The purpose of this page is to show the steps to create a simple webapp running as a docker container in Azure.

Create Container Registry

I am using a new Azure Container Registry in my resource group called SecuredContainerRegistry which I will refer to throughout this page. I have created this using a Basic SKU which is sufficient enough for this purpose. If you need a private endpoint you need to change he SKU to premium.

"resources": [ { "type": "Microsoft.ContainerRegistry/registries", "apiVersion": "2020-11-01-preview", "name": "SecuredContainerRegistry", "location": "[resourceGroup().location]", "dependsOn": [], "tags": "[variables('tagsArray')]", "sku": { "name": "Basic", "tier": "Basic" }, "properties": { "adminUserEnabled": true, "publicNetworkAccess": "Enabled", "zoneRedundancy": "Disabled" } }

The main change after creating the Container Registry using Default options is to enable Admin user which allows us to login used by docker.

Next important change we do on this resource is to register this resource in AAD by giving it a System assigned Identity using the portal.

Add Service connection

Next you need to add a service connection in your Azure Devops project using service principal authentication that to get access to Azure Container Registry. In the popup select Azure Container registery as Registry type, then select your azure container registry and give the service connection a name.

Buid and Deploy Docker project

Create a .net core application including a Dockerfile for windows. When you choose in Visual Studio it generates a Dockerfile which is not completely working. The following example is changed version of that file which works fine:

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
#Depending on the operating system of the host machines(s) that will build or run the containers, the image specified in the FROM statement may need to be changed.
#For more information, please see https://aka.ms/containercompat

FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["*.csproj", "SecuredWebApi/"]
RUN dotnet restore "SecuredWebApi/SecuredWebApi.csproj"
WORKDIR "/src/SecuredWebApi"
COPY . .
RUN dotnet build "SecuredWebApi.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "SecuredWebApi.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "SecuredWebApi.dll"]

Create Container Instance

Next step is to create an Azure Container instance. In my example I gave it the name container-instance. During the creation you need to connect this to a container registry. You can choose Azure Container Registry created in the first step above. Once you have a successful build you will have an image available to associate it with it. Based on my sample project I named this instance secured-container-instance and Azure portal automatically recognizes the container registery when you select the Image Source from Azure Container Registry. For this project I exposed ports 80 and port 443

"resources": [ { "location": "westeurope", "name": "secured-container-instance", "type": "Microsoft.ContainerInstance/containerGroups", "apiVersion": "2021-03-01", "properties": { "containers": [ { "name": "secured-container-instance", "properties": { "image": "securedcontainerregistry.azurecr.io/securedazurelib:latest", "resources": { "requests": { "cpu": "1", "memoryInGB": "1.5" } }, "ports": [ { "protocol": "TCP", "port": 80 }, { "protocol": "TCP", "port": 443 } ], } } ], "restartPolicy": "[parameters('restartPolicy')]", "osType": "Linux", "imageRegistryCredentials": [ { "server": "securedcontainerregistry.azurecr.io", "username": "[parameters('imageUsername')]", "password": "[parameters('imagePassword')]" } ], "ipAddress": { "type": "Public", "ports": "80 (TCP), 443 (TCP)" } }, "tags": {} }

Start the instance

By starting the instance on Container the image gets pulled and deployed in the container instance. On the Azure Portal you can look into the public ip adress and check the website is running and accessible.

A container instance can be started using docker command: docker run securedcontainerregistry.azurecr.io/securedazurelib:latest The following job tasks will start an instance in an Azure pipeline

jobs: - job: RunTest workspace: clean: all pool: vmImage: 'ubuntu-latest' steps: - task: Docker@2 displayName: Login to ACR inputs: command: login containerRegistry: securedcontainerregistry - script: | docker run securedcontainerregistry.azurecr.io/somerepo/securedazurelib:latest

Security

You can register the container instance in AAD using Manage Identity and then assign a role in KeyVault for that identity to allow access to secrets.

Monday, May 3, 2021

Secrets in ARM templates

 You may want to use a password in a template (let’s say user password of a VM or admin password of a SQL-server). Putting the password in your template, which is located in your source code repository, is not according to security guidelines.

One option to secure your strings would be to put them in KeyVault as a Secret and refer them from either paramters.json or in your main.json where it refer to a linked template.

First you need to set Azure Resource Manager for template deployment on checked within Access Policies of the keyvault where the template is refering to.

Deploying Templates

You could start a deployment right from the portal by adding a resource of type Template Deployment.

Another option would be using az CLI:

az deployment group create --resource-group newgrp1 --template-file main.json --parameters parameters.json

Or you could deploy it from your CD-pipeline locatedin Azure Devops.

Using Secured Secrets in Parameters.json

The following example refers to a secret called vmpassword within a keyvault called demovault10001 ie. located in newgrp1 resource group:

{ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", "contentVersion": "1.0.0.0", "parameters": { "adminUsername": { "value": "admin" }, "adminPassword": { "reference": { "keyVault": { "id": "/subscriptions/baaa99b3-1d19-4c5e-90e1-39d55de5fc6e/resourceGroups/newgrp1/providers/Microsoft.KeyVault/vaults/demovault10001" }, "secretName": "vmpassword" } } } }

Using Secured Secrets in main.json

Similar to above example we can refer to a secured password by setting the keyvault id and the secret name. In the following example we use this to pass the adminPassword as a parameter to nested template.

 { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "location": { "type": "string", "defaultValue": "[resourceGroup().location]", "metadata": { "description": "The location where the resources will be deployed." } }, "vaultName": { "type": "string", "defaultValue":"appvault10001" }, "secretName": { "type": "string", "defaultValue":"vmaccountpassword" }, "vaultResourceGroupName": { "type": "string", "defaultValue":"newgrp1" }, "vaultSubscription": { "type": "string", "defaultValue": "[subscription().subscriptionId]", "metadata": { "description": "The name of the subscription that contains the keyvault." } } }, "resources": [ { "type": "Microsoft.Resources/deployments", "apiVersion": "2018-05-01", "name": "dynamicSecret", "properties": { "mode": "Incremental", "expressionEvaluationOptions": { "scope": "inner" }, "template": { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "adminLogin": { "type": "string" }, "adminPassword": { "type": "securestring" }, "location": { "type": "string" } }, "variables": { "sqlServerName": "[concat('sql-', uniqueString(resourceGroup().id, 'sql'))]" }, "resources": [ { "type": "Microsoft.Sql/servers", "apiVersion": "2018-06-01-preview", "name": "[variables('sqlServerName')]", "location": "[parameters('location')]", "properties": { "administratorLogin": "[parameters('adminLogin')]", "administratorLoginPassword": "[parameters('adminPassword')]" } } ], "outputs": { "sqlFQDN": { "type": "string", "value": "[reference(variables('sqlServerName')).fullyQualifiedDomainName]" } } }, "parameters": { "location": { "value": "[parameters('location')]" }, "adminLogin": { "value": "demousr" }, "adminPassword": { "reference": { "keyVault": { "id": "[resourceId(parameters('vaultSubscription'), parameters('vaultResourceGroupName'), 'Microsoft.KeyVault/vaults', parameters('vaultName'))]" }, "secretName": "[parameters('secretName')]" } } } } } ], "outputs": { } }

Friday, January 22, 2021

Apply SSL to your web service

You can chose to use a free managed certificate for your web app from Azure Portal:

  1. In Azure Portal go to TLS/SSL settings blade of your webapp.

  2. Choose Private Key Certificates on the top

  3. Click on + Create App Service Managed Certificate

  4. In the popup select the sub-domain you want to choose

  5. Wait… and the sub-domain will be verified and will be get green:


  6. Hit on Create button and wait until it is finished.

  7. Go to Custom domains blade in your App service.

  8. Notice that the newly created certificates are part of the custom domains.




  9. Hit on Add binding on the sub-domain you want to associate.

  10. In the popup select the certificate and SSL Type. Then hit Add button.


 

Thursday, December 10, 2020

Building Data Science Team

 From the book: http://www.datascienceassn.org/sites/default/files/Building Data Science Teams.pdf

A data-driven organization acquires, processes, and leverages data in a timely fashion to create efficiencies, iterate on and develop new products, and navigate the competitive landscape.

Author wrote also:

There are many ways to assess whether an organization is data driven. Some like to talk about how much data they generate. Others like to talk about the sophistication of data they use, or the process of internalizing data. I prefer to start by highlighting organizations that use data effectively.


See also: altexsoft.com blog post on datascience: how-to-structure-data-science-team-key-models-and-roles

Friday, July 24, 2020

What was my Wifi password?

If you have a Wifi connection on your Laptop and you need to know the password for it follow the next steps in a DOS command line:

netsh wlan show profile

This will show the known profiles. Copy the name of any profile you need to know its password.
Then call the next command to see the password in clear text:

netsh wlan show profile <name> key=clear