A day with .Net

My day to day experince in .net

SignalR Communication In Multiple Projects (Multiple Server Hubs) – Micro-services Architecture

Posted by vivekcek on December 15, 2018

I have an MVC web application that communicate to a back-end web API. The API is not exposed directly to public and resides inside a virtual network.
But I want to implement a SignalR connection between my front-end MVC and back end web api.
In this setup I have two SignalR clients and two SignalR servers.
The first SignalR client is a JavaScript client that communicate to the SignalR server in the MVC application.
The second SignalR client is a .NET SignalR client inside the MVC app that can connect to SignalR server in the API.

The setup works this way.
1.JavaScript client establish a connection with MVC SignalR server hub.
2.The MVC SiganlR server hub establish a connection with API server hub through the SignalR .NET client inside the MVC app.

This is my JavaScript Client.

"use strict";

var connection = new signalR.HubConnectionBuilder().withUrl("/mvcHub").build();

connection.on("ReceiveMessage", function (user, message) {
    var msg = message.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
    var encodedMsg = user + " says " + msg;
    var li = document.createElement("li");
    li.textContent = encodedMsg;
    document.getElementById("messagesList").appendChild(li);
});

connection.start().catch(function (err) {
    return console.error(err.toString());
});

document.getElementById("sendButton").addEventListener("click", function (event) {
    var user = document.getElementById("userInput").value;
    var message = document.getElementById("messageInput").value;
    connection.invoke("SendMessage", user, message).catch(function (err) {
        return console.error(err.toString());
    });
    event.preventDefault();
});

This is my MVC Server hub

using System.Threading.Tasks;
using Microsoft.AspNetCore.SignalR;
using Microsoft.AspNetCore.SignalR.Client;

namespace WebApplication2
{
    public class MvcHub:Hub
    {
        private readonly IHubContext<MvcHub> _hubContext;
        public MvcHub(IHubContext<MvcHub> hubContext)
        {
            _hubContext = hubContext;
        }
        public async Task SendMessage(string user, string message)
        {

            var connection = new HubConnectionBuilder()
                .WithUrl("http://localhost:50735/apiHub")
                .Build();
            connection.StartAsync().Wait();

            await connection.InvokeAsync("SendMessage",
                    "sadsd", "sdf");
            
            connection.On<string, string>("ReceiveMessage", (user1, message1) =>
            {
                _hubContext.Clients.All.SendAsync("ReceiveMessage", user1, message1);
            });
        }
    }
}

This is my API Server hub

using System.Threading.Tasks;
using Microsoft.AspNetCore.SignalR;
namespace WebApplication1
{    public class ApiHub : Hub
    {
        public async Task SendMessage(string user, string message)
        {
            await Clients.Caller.SendAsync("ReceiveMessage", user, message);
        }
    }
}

From inside your API controller you can route message to MVC UI via this code.

public class ValuesController : ControllerBase
    {
        private readonly IHubContext<ApiHub> _hubContext;
        public ValuesController(IHubContext<ApiHub> hubContext)
        {
            _hubContext = hubContext;

        }

        // GET api/values/5
        [HttpGet("{id}")]
        public ActionResult<string> Get(int id)
        {
            _hubContext.Clients.All.SendAsync("ReceiveMessage", "vivek", "asdad");
            return "value";
        }
    }

Advertisements

Posted in SignalR | Tagged: , , , | Leave a Comment »

The Democratization Of Machine Learning With ML.NET – Linear Regression With Gradient Boosted Tree (Ensemble learning method)

Posted by vivekcek on September 26, 2018

NB: This post is based on ML.NET 0.4 version. There will be some changes in 0.5 version.

Microsoft has decided to make Machine Learning accessible to .NET developers through ML.NET.
ML.NET API is very simple and easy to learn. You can forget about large mathematical equations and implementations of algorithms.

Below diagram show the process of building models using ML.NET.

1. Prepare your data.
2. Decide the algorithm to use.
3. Train a model using the selected algorithm.
4. Evaluate the model.
5. Save the model.
6. Use the saved model in your .NET applications.

1

Below diagram show the main components of ML.NET.

1. Transforms : – Use when you want to transform your data, like for converting categorical values to numerical vectors.
2. Learners : – Consists of various algorithms. Select one that is a perfect fit for your data and problem.
3. Misc :- Consists of API’s for loading data, doing evaluation’s etc..

2

ML.NET follow a pipeline kind of architecture. The pipeline architecture is shown below.

In this post, i will implement linear regression with ML.NET.The problem we are going to solve is predicting the New York taxi fare.
Data is divided into Training set and Testing set.Download it from here.

Train Data
Test Data

Our data include the below columns. The last column “fare_amount” is our label(value to be predicted).

3

ML.NET is cross platform that means you can do this in any OS that has .NET Core 2.0 installed.You can use either VS Code or Visual Studio. I am planning to use VS Code and .NET Core CLI.

Follow below steps(Completed source code will be provided at the end).

1. Make sure .NET Core 2.0 SDK is installed.

2. Create a .NET Core console app via command prompt.

>dotnet new console -o myApp
>cd myApp

3. Install ML.NET package

dotnet add package Microsoft.ML --version 0.4.0

4. Download and save the train and test dataset from above link.

5. Include below namespaces.

using System;
using System.IO;
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Models;
using Microsoft.ML.Trainers;
using Microsoft.ML.Transforms;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Api;

6. Create two clasees that represent our input and output.

    public class TaxiTrip
    {
        [Column("0")]
        public string VendorId;
        [Column("1")]
        public string RateCode;
        [Column("2")]
        public float PassengerCount;
        [Column("3")]
        public float TripTime;
        [Column("4")]
        public float TripDistance;
        [Column("5")]
        public string PaymentType;
        [Column("6")]
        public float FareAmount;
    }

    public class TaxiTripFarePrediction
    {
        [ColumnName("Score")]
        public float FareAmount;
    }

7. Declare the data path. Store your csv files inside a folder named “Data”.

static readonly string _datapath = Path.Combine(Environment.CurrentDirectory, "Data", "taxi-fare-train.csv");
static readonly string _testdatapath = Path.Combine(Environment.CurrentDirectory, "Data", "taxi-fare-test.csv");
static readonly string _modelpath = Path.Combine(Environment.CurrentDirectory, "Data", "Model.zip");

8. Declare learning pipeline and load data.

var pipeline = new LearningPipeline();
pipeline.Add(new TextLoader(_datapath).CreateFrom<TaxiTrip>(useHeader: true, separator: ','));

In this step the data will be stored in memory like below.

4

9. Copy the FareAmount to Label column.

pipeline.Add(new ColumnCopier(("FareAmount", "Label")));

In this step a new column(Label) will be added to our data by copying the values of FareAmount.

5

10. Convert string values to numerical vectors. Beacause our algorithm only support numerical vectors.

pipeline.Add(new CategoricalOneHotVectorizer("VendorId",
                                             "RateCode",
                                             "PaymentType"));

After this step data will be look like below. String values are now converted to numerical vectors.

6

11. Now copy all of our input coulmns to a single coulmn named Features.

pipeline.Add(new ColumnConcatenator("Features",
                                    "VendorId",
                                    "RateCode",
                                    "PassengerCount",
                                    "TripDistance",
                                    "PaymentType"));

In this step a new column(Features) will be added to our data structure.

7

12. Next is the important step of selecting the Algorithm. In our case i am using FastTreeRegressor algorithm.
Which is a Gradient Boosted Decesion Tree based algorithm, which provide better performance on non linear dataset.

pipeline.Add(new FastTreeRegressor());

13. Now we train our model.Our algorithm will use Features and Label column.

PredictionModel<TaxiTrip, TaxiTripFarePrediction> model = pipeline.Train<TaxiTrip, TaxiTripFarePrediction>();

So what we have done still now can be represented as below.

14. Once you trained the model you can evaluate its performance by checking RSquared. If RSquared is close to 1, our model perform good.

var testData = new TextLoader(_testdatapath).CreateFrom<TaxiTrip>(useHeader: true, separator: ',');

var evaluator = new RegressionEvaluator();
RegressionMetrics metrics = evaluator.Evaluate(model, testData);

Console.WriteLine($"Rms = {metrics.Rms}");
Console.WriteLine($"RSquared = {metrics.RSquared}");

8

15 Now predict a new value as below.

TaxiTripFarePrediction prediction = model.Predict(TestTrips.Trip1);
Console.WriteLine("Predicted fare: {0}, actual fare: 29.5", prediction.FareAmount);

9

16. Where TestTrips is a static class.


    static class TestTrips
    {
        internal static readonly TaxiTrip Trip1 = new TaxiTrip
        {
            VendorId = "VTS",
            RateCode = "1",
            PassengerCount = 1,
            TripDistance = 10.33f,
            PaymentType = "CSH",
            FareAmount = 0 // predict it. actual = 29.5
        };
    }

17. To save and retrieve your model use below codes.

 await model.WriteAsync(_modelpath);
PredictionModel<TaxiTrip, TaxiTripFarePrediction> model = await PredictionModel.ReadAsync<TaxiTrip, TaxiTripFarePrediction>(_modelpath);

18. Full Source Code is given below.

using Microsoft.ML;
using Microsoft.ML.Data;
using System;
using System.IO;
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Models;
using Microsoft.ML.Trainers;
using Microsoft.ML.Transforms;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Api;

namespace LinearRegression
{

    class Program
    {

        static readonly string _datapath = Path.Combine(Environment.CurrentDirectory, "Data", "taxi-fare-train.csv");
        static readonly string _testdatapath = Path.Combine(Environment.CurrentDirectory, "Data", "taxi-fare-test.csv");
        static readonly string _modelpath = Path.Combine(Environment.CurrentDirectory, "Data", "Model.zip");

        static void Main(string[] args)
        {
            var pipeline = new LearningPipeline();
            pipeline.Add(new TextLoader(_datapath).CreateFrom<TaxiTrip>(useHeader: true, separator: ','));
            pipeline.Add(new ColumnCopier(("FareAmount", "Label")));
            pipeline.Add(new CategoricalOneHotVectorizer("VendorId",
                                             "RateCode",
                                             "PaymentType"));

            pipeline.Add(new ColumnConcatenator("Features",
                                    "VendorId",
                                    "RateCode",
                                    "PassengerCount",
                                    "TripDistance",
                                    "PaymentType"));

            pipeline.Add(new FastTreeRegressor());

            PredictionModel<TaxiTrip, TaxiTripFarePrediction> model = pipeline.Train<TaxiTrip, TaxiTripFarePrediction>();
            //await model.WriteAsync(_modelpath);
            //PredictionModel<TaxiTrip, TaxiTripFarePrediction> model = await PredictionModel.ReadAsync<TaxiTrip, TaxiTripFarePrediction>(_modelpath);

            var testData = new TextLoader(_testdatapath).CreateFrom<TaxiTrip>(useHeader: true, separator: ',');

            var evaluator = new RegressionEvaluator();
            RegressionMetrics metrics = evaluator.Evaluate(model, testData);

            Console.WriteLine($"Rms = {metrics.Rms}");
            Console.WriteLine($"RSquared = {metrics.RSquared}");

            TaxiTripFarePrediction prediction = model.Predict(TestTrips.Trip1);
            Console.WriteLine("Predicted fare: {0}, actual fare: 29.5", prediction.FareAmount);

            Console.Read();
        }
    }

    public class TaxiTrip
    {
        [Column("0")]
        public string VendorId;
        [Column("1")]
        public string RateCode;
        [Column("2")]
        public float PassengerCount;
        [Column("3")]
        public float TripTime;
        [Column("4")]
        public float TripDistance;
        [Column("5")]
        public string PaymentType;
        [Column("6")]
        public float FareAmount;
    }

    public class TaxiTripFarePrediction
    {
        [ColumnName("Score")]
        public float FareAmount;
    }

    // <Snippet1>
    static class TestTrips
    // </Snippet1>
    {
        // <Snippet2>
        internal static readonly TaxiTrip Trip1 = new TaxiTrip
        {
            VendorId = "VTS",
            RateCode = "1",
            PassengerCount = 1,
            TripDistance = 10.33f,
            PaymentType = "CSH",
            FareAmount = 0 // predict it. actual = 29.5
        };
        // </Snippet2>
    }
}

Posted in Machine Learning | Leave a Comment »

Serverless meet Containerization – Running Azure functions in Docker and then to Kubernetes.

Posted by vivekcek on June 23, 2018

Microservices, Containerization and Serverless are the current hot architecture patterns in our industry.Each have their pros and cons. I will disuss about the selection criteria for these in a later post.

In this post i am trying to bring the Microsoft serverless platform Azure functions inside the Docker Container.

1. Make sure you installed Docker for windows Edge.
2. Switch to Linux containers. Azure functions now support only linux images.
3. Install Azure Functions CLI Version 2.0 from this link https://docs.microsoft.com/en-us/azure/azure-functions/functions-run-local
4. After installing the CLI open command prompt and issue below command.

func init MyFunctionProject --docker

5. Select dotnet

6. Now switch to MyFunctionProject folder and issue below command. Select HTTP Triger as template.

func new

7.Now issue below command to test whether your function work fine locally.

func host start

8. Your function will run at http://localhost:7071/api/myfunction?name=”vivek&#8221;
9. Now add a Docker file.

FROM microsoft/azure-functions-base:dev-nightly
ENV AzureWebJobsScriptRoot=/home/site/wwwroot
COPY . /home/site/wwwroot

10. Now build the Docker Image.

docker build -t functionapp .

11. Now run the image.

docker run -p 8080:80 functionapp

12. Your function will be running at http://localhost:8080/api/myfunction?name=”vivek&#8221;

4. Once the image is running successfully we can deploy the image to kubernetes.
5. First create a deployment.

kubectl run hello --image=functionapp:latest --image-pull-policy=IfNotPresent

6. Now create a service to expose the deployment aka pod.

kubectl expose deployment hello --type="LoadBalancer" --port=8080 --target-port=80

7.Now browse to http://localhost:8080/api/myfunction?name=”vivek&#8221;

Posted in Azure, Docker, Kubernetes | Leave a Comment »

Deploying a local ASP.NET Core docker image to kubernetes from your local machine (No docker registry)

Posted by vivekcek on June 23, 2018

I faced some issues in deploying a local asp.net core image to my local kubernetes cluster.So in this post I will describe the steps to deploy a local docker image to kubernetes cluster.Hope you installed Docker For Windows Edge version and Enabled Kubernetes support, Also switch to Linux containers.

1. First build a local asp.net core docker image from below docker file. Use linux container.

FROM microsoft/aspnetcore-build:2.0 AS build-env
WORKDIR /app
EXPOSE 80

WORKDIR /app

# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore 

# Copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out

# Build runtime image
FROM microsoft/aspnetcore:2.0
WORKDIR /app
COPY --from=build-env /app/out .

ENTRYPOINT ["dotnet", "KubeTest.dll"]

2. Now build your image.

docker build -t mycoreapp .

3. Now run your image. This step is optional, just verify your image is working fine.

docker run -p 8080:80 mycoreapp

4. Once the image is running successfully we can deploy the image to kubernetes.
5. First create a deployment.

kubectl run hello --image=mycoreapp:latest --image-pull-policy=IfNotPresent

6. Now create a service to expose the deployment aka pod.

kubectl expose deployment hello --type="LoadBalancer" --port=8080 --target-port=80

7.Now browse to http://localhost:8080/

Posted in Docker, Kubernetes | Tagged: | Leave a Comment »

Installing and Uninstalling of Kubernetes Dashboard – Docker for windows

Posted by vivekcek on June 17, 2018

I installed the Docker for windows edge with Kubernetes enabled(Switch to linux containers to use kubernetes with docker for windows edge).
After deploying a pod to kubernetes, I found that the kubernetes dashboard is missing. So i tried to install it from various sources.
End of the day my dashboard installation got corrupted and I had to re-install it again.

In this post, I will show you how to install kubernetes dashboard first, then how to uninstall and reinstall.

1. Download and install Docker for windows edge.
2. Enable kubernetes.
3. Docker for windows is not coming with kubernetes dashboard. So we need to install it manually.
4. Run Docker as administrator.
5. Open command prompt and enter below command.

kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

6. After installation run below command.

kubectl proxy

7. Now open the browser and paste below url.

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.

How to uninstall dashboard.

Execute below commands, one by one. Then follow the above installation steps 5 to 7.

kubectl delete deployment kubernetes-dashboard --namespace=kube-system 
kubectl delete service kubernetes-dashboard  --namespace=kube-system 
kubectl delete role kubernetes-dashboard-minimal --namespace=kube-system 
kubectl delete rolebinding kubernetes-dashboard-minimal --namespace=kube-system
kubectl delete sa kubernetes-dashboard --namespace=kube-system 
kubectl delete secret kubernetes-dashboard-certs --namespace=kube-system
kubectl delete secret kubernetes-dashboard-key-holder --namespace=kube-system

Posted in Docker, Kubernetes, kubernetes dashboard instllation | Tagged: , , , | Leave a Comment »

Connecting to local or remote SQL Server from Docker Container

Posted by vivekcek on June 10, 2018

In this post, I will show you how to connect from a docker container to local sql express running in your machine.
Please note that the SQL Server is not running inside a docker container. SQL is installed in my machine.

My docker container host an asp.net core application with EF Core. During the local debugging the app was perfectly connecting to my sql database. But after deploying the app to docker container I got exception like unable to connect to sql server.

So why this happen?. Here the docker container is running as a separate machine inside your host computer. So to connect to the sql database in your host machine, you need to enable remote connections to sql server.

So follow below steps to enable remote connections to sql server.

1. You must configure SQL Server with Mixed Mode Authentication. For remote connection you need to supply user name and password.
2. Open SQL Server Configuration Manager as Administrator.

3.Now select Protocols for SQL and then select TCP/IP now open the Properties by right click.
4.In the TCP/IP properties window enable TCP/IP for SQL Server (Enabled=Yes, Listen All=Yes).

5.Now select IP Addresses tab in properties window. And under IPAll give port number as 49172.

6. Now restrat SQL Server service.

7.Make sure SQL Server Browser is running, if not start it.

7. Now setup your firewall to accept inbound connection to 49172.

8.To connect to sql server from docker, you can’t use the host computer name. You need to find the right ip address of you host.

9. Open command prompt and issue “ipconfig” command, Then you can see a nat ip address of Docker NAT ip address copy it.

10. Now we need to make sure, we can connect to this ip from our docker container.
11.User “docker ps” command to find the id of your running container.
12. Now use the “docker exec -i cmd” to get command prompt from your container. And use the ping command to chek the ip address connectivity.

13. If the connectivity is fine you can use below connection string.

Data Source=<IPAddress>\\SQLEXPRESS,49172;Database=<Database name>;User ID=sa;Password=<Your password>;MultipleActiveResultSets=true;

Posted in Docker | Tagged: , , , | Leave a Comment »

GDPR(General Data Protection Regulation) rules for Software Developers and Architects

Posted by vivekcek on May 29, 2018

As a user I love GDPR. I own my data. No one can use my data for their advantage.
But as a software professional, I am concerned about the systems we are developing.
So to address the concerns of Software professionals, I am writing this blog.
I am going to write about some of the rules you need to implement in your systems to agree GDPR.

Hey what the @&** is this GDPR? I don’t want to follow.

If you don’t want to follow GDPR this is going to happen “Penalties and fines can be as high as 4% of annual revenue or €20 million, whichever is greater”.

So better to follow GDPR, I can help you.

What is GDPR?

Its a set of guidelines you need to implement in your systems to protect users personal or simply user data. User is the king. Even if you own the software system, User has the complete ownership of his data.

So What kind of data?

Any data that identify the user.
Name, Location, Health, Genetic, Social handles and more.
Other than that the data created by user like Orders, Messages, Tweets etc..

Is my website need to implement GDPR? Yes, You should if.

Your website collects data on visitors, such as via Google analytics.
Your site has a registration form.
You have e-commerce functionality on your site; that is, you collect information to process payments, orders etc…
You have a newsletter sign-up form.
You include social media links on your pages e.g. Facebook, Twitter etc…
You use a comments system for your articles, such as Disqus.
Your site has scripts that use cookies.
You have a contact form for users to get in touch.

So What are the rules?. The rules are not from actual regulation, I just grouped some guidelines as rules.

Rule 1

You should only collect data necessary for your business.

Rule 2

Data at rest and data in transmit must be encrypted.
Good encryption algorithm need to be used.
Protect your encryption key in a secure place.
Use SSL for data communication.
Encrypt your backups.
You should inform user their data is stored by encrypting.

Rule 3

Your sessions and cookies must expire.
Make sure disabled users can’t use still to expire cookies.
Perfect authentication token revocation should be implemented.

Rule 4

Never try to track users.

Rule 5

If you store users IP or location, it must be informed

Rule 6

Implement Password complexity and store as salted SHA-256 hash.

Rule 7

Avoid personal security questions, ask users to create custom questions that is not related to their personal things.
Use 2 factor authentication.

Rule 8

Data sharing with 3rd parties must be informed to users.
Like to splunk, google analytics, Azure.

Rule 9

Hacking attempts or breach must be informed to users and government.

Rule 10

User must be able to delete all the data generated by him.
Do a Cascade delete.
Use Nullable foreign key for userid.
Keep only user id while delete, anonymize name, email etc..
Can have a background job to delete.

Rule 11

User must able to map him as restricted.
Back office staff can’t see restricted users data.
Restricted users must not be listed in search results.
Add a Boolean column in your database against user to mark him as restricted.

Rule 12

User must be able to export all his data personnel orders messages etc.
Export format can be JSON, XML,CSV etc.
Can be exported to other vendor.
Can be manual like twitter, send an email later.
May be a background job.

Rule 13

User must be able to edit all his personal info.
After change re validate email , phone if needed.

Rule 14

Ask for consent.
Checkbox for each data processing purpose.
if using for machine learning get consent.
Able to withdraw consent.
Consent management location.
Request new consent via email or app popup.

Rule 15

User must be able to see data

Rule 16

Age check, above 18?
Can be violated do your part.

Rule 17

Anonymous data, don’t use production data in test staging.

Rule 18

Data integrity via checksum, audit trail.

Rule 19

Log access to personal data.
Do not put personnel data in blockchain
Forgotten users id need to keep in separate, because during backup restore data should not come back
Data retention- delete data after some time, let the user decide.

Posted in GDPR | Tagged: , , | Leave a Comment »

ASP.NET Core Server Side And Client Side Exception Logging to SQL Server By Using Serilog and JSNLog

Posted by vivekcek on May 28, 2018

Server side and client side logging is a must feature for most of the production web application.
For server side logging i usually prefer ELMAH, But nowadays they started pricing. So i am going with Serilog.
For client side logging we can use JSNLog.
Due to lack of time i am not going to write a detailed post. But if you follow the steps you are done.
Please note these steps are for ASP.NET Core 2.0 web apps.

1. Create an ASP.NET Core 2.0 Web App.
2. Install the below dependencies.

serilog.aspnetcore

Serilog.Settings.Configuration

Serilog.Sinks.MSSqlServer

JSNLog.AspNetCore

Destructurama.JsonNet

3. Now add two classes CustomLoggingAdapter and LogMessageHelpers as below

using JSNLog;
using JSNLog.Infrastructure;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;

namespace Logging
{
    public class CustomLoggingAdapter : ILoggingAdapter
    {
        private ILoggerFactory _loggerFactory;

        public CustomLoggingAdapter(ILoggerFactory loggerFactory)
        {
            _loggerFactory = loggerFactory;
        }

        public void Log(FinalLogData finalLogData)
        {
            ILogger logger = _loggerFactory.CreateLogger(finalLogData.FinalLogger);

            Object message = LogMessageHelpers.DeserializeIfPossible(finalLogData.FinalMessage);

            switch (finalLogData.FinalLevel)
            {
                case Level.TRACE: logger.LogTrace("{@logMessage}", message); break;
                case Level.DEBUG: logger.LogDebug("{@logMessage}", message); break;
                case Level.INFO: logger.LogInformation("{@logMessage}", message); break;
                case Level.WARN: logger.LogWarning("{@logMessage}", message); break;
                case Level.ERROR: logger.LogError("{@logMessage}", message); break;
                case Level.FATAL: logger.LogCritical("{@logMessage}", message); break;
            }
        }
    }

    internal class LogMessageHelpers
    {
        public static T DeserializeJson<T>(string json)
        {
            T result = JsonConvert.DeserializeObject<T>(json);
            return result;
        }
        public static bool IsPotentialJson(string msg)
        {
            string trimmedMsg = msg.Trim();
            return (trimmedMsg.StartsWith("{") && trimmedMsg.EndsWith("}"));
        }

        /// <summary>
        /// Tries to deserialize msg.
        /// If that works, returns the resulting object.
        /// Otherwise returns msg itself (which is a string).
        /// </summary>
        /// <param name="msg"></param>
        /// <returns></returns>
        public static Object DeserializeIfPossible(string msg)
        {
            try
            {
                if (IsPotentialJson(msg))
                {
                    Object result = DeserializeJson<Object>(msg);
                    return result;
                }
            }

            catch
            {
            }
            return msg;

        }

        /// <summary>
        /// Returns true if the msg contains a valid JSON string.
        /// </summary>
        /// <param name="msg"></param>
        /// <returns></returns>
        public static bool IsJsonString(string msg)
        {
            try
            {
                if (IsPotentialJson(msg))
                {
                    // Try to deserialise the msg. If that does not throw an exception,
                    // decide that msg is a good JSON string.
                    DeserializeJson<Dictionary<string, Object>>(msg);
                    return true;
                }
            }
            catch
            {
            }
            return false;
        }
        /// <summary>
        /// Takes a log message and finds out if it contains a valid JSON string.
        /// If so, returns it unchanged.
        /// 
        /// Otherwise, surrounds the string with quotes (") and escapes the string for JavaScript.
        /// </summary>
        /// <param name="ms"></param>
        /// <returns></returns>
        public static string EnsureValidJson(string msg)
        {
            if (IsJsonString(msg))
            {
                return msg;
            }
            return HtmlHelpers.JavaScriptStringEncode(msg, true);
        }

    }
}

4. Now update your Program.cs as below.

using System;
using System.IO;
using Destructurama;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Serilog;

namespace Logging
{
    public class Program
    {
        public static IConfiguration Configuration { get; } = new ConfigurationBuilder()
             .SetBasePath(Directory.GetCurrentDirectory())
             .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
             .AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "Production"}.json", optional: true)
             .Build();

        public static void Main(string[] args)
        {
            Log.Logger = new LoggerConfiguration()
                .ReadFrom.Configuration(Configuration)
                .Destructure.JsonNetTypes()
                .CreateLogger();

            BuildWebHost(args).Run();
        }

        public static IWebHost BuildWebHost(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseStartup<Startup>()
                .UseSerilog()
                .Build();
    }
}

5. Update your appSettings.json. You need a SQL database. Logs table will be created automatically.

{
  "connectionStrings": {
    "libraryDBConnectionString": "Server=SQLEXPRESS;Database=Test;Trusted_Connection=True;MultipleActiveResultSets=true;"
  },
  "Serilog": {
    "MinimumLevel": "Error",
    "WriteTo": [
      {
        "Name": "MSSqlServer",
        "Args": {
          "connectionString": "Server=SQLEXPRESS;Database=Test;Trusted_Connection=True;MultipleActiveResultSets=true;",
          "tableName": "Logs",
          "autoCreateSqlTable": true
        }
      }
    ]
  }
}

6. Now add a global extension class AspNetCoreGlobalExceptionHandlerExtension

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Diagnostics;
using Microsoft.Extensions.Logging;
using System;

namespace Logging
{
    public static class AspNetCoreGlobalExceptionHandlerExtension
    {
        public static IApplicationBuilder UseAspNetCoreExceptionHandler(this IApplicationBuilder app)
        {
            var loggerFactory = app.ApplicationServices.GetService(typeof(ILoggerFactory)) as ILoggerFactory;
            return app.UseExceptionHandler(HandleException(loggerFactory));
        }

        public static Action<IApplicationBuilder> HandleException(ILoggerFactory loggerFactory)
        {
            return appBuilder =>
            {
                appBuilder.Run(async context =>
                {
                    var exceptionHandlerFeature = context.Features.Get<IExceptionHandlerFeature>();
                    if (exceptionHandlerFeature != null)
                    {
                        var logger = loggerFactory.CreateLogger("Serilog Global exception logger");
                        logger.LogError(500, exceptionHandlerFeature.Error, exceptionHandlerFeature.Error.Message);

                    }

                    context.Response.Redirect($"/Error/Status/{context.Response.StatusCode}");

                });

            };

        }
    }
}

7. Update your Startup.cs

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseBrowserLink();
                app.UseDeveloperExceptionPage();
                app.UseAspNetCoreExceptionHandler();
            }
            else
            {
                app.UseAspNetCoreExceptionHandler();
            }

            app.UseStaticFiles();

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
            });
        }

8. Inject logger in your HomeController and throw an exception.

using System;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;

namespace Logging.Controllers
{
    public class HomeController : Controller
    {
        private ILogger<HomeController> _logger;

        public HomeController(ILogger<HomeController> logger)
        {
            _logger = logger;
        }

        public IActionResult Index()
        {
            try
            {
                throw new InvalidOperationException("Exception message");
            }
            catch (ArgumentNullException e)
            {
                _logger.LogError(e.Message, e.InnerException);
                return View("Index");
            }
        }

    }
}

9. Now go to your database table and see the logged exception.

10. Next we will configure JSNLog for client side logging.

11. Add below code in _Layout.cshtml

@*Add to _Layout.cshtml*@
<script src="https://cdnjs.cloudflare.com/ajax/libs/jsnlog/2.26.2/jsnlog.min.js"></script>

12. Update your Startup.cs

using JSNLog;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

namespace Logging
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }
     
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc();
        }

     
        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
            if (env.IsDevelopment())
            {
                app.UseBrowserLink();
                app.UseDeveloperExceptionPage();
                app.UseAspNetCoreExceptionHandler();
            }
            else
            {
                app.UseAspNetCoreExceptionHandler();
            }
            // Configure JSNLog
            var jsnlogConfiguration = new JsnlogConfiguration();
            app.UseJSNLog(new LoggingAdapter(loggerFactory), jsnlogConfiguration);

            app.UseStaticFiles();

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
            });
        }
    }
}

13. In the View you can write below codes for logging javascript exceptions and unhandled exceptions.

<script src="https://cdnjs.cloudflare.com/ajax/libs/jsnlog/2.26.2/jsnlog.min.js"></script>
    <script type="text/javascript">

        JL().error("log message");

        // Code included in jsnlog.js to set a handler that logs uncaught JavaScript errors to 
        // the server side log.
     
        if (window && !window.onerror) {
            window.onerror = function (errorMsg, url, lineNumber, column, errorObj) {
                JL("onerrorLogger").fatalException({
                    "msg": "Uncaught Exception",
                    "errorMsg": errorMsg, "url": url,
                    "line number": lineNumber, "column": column
                }, errorObj);

                return false;
            }
        }
    </script>

14. Now see the result in your database.

15. To do the same in WebApi, use below extension.

public static class ApiGlobalExceptionHandlerExtension
    {
        public static IApplicationBuilder UseWebApiExceptionHandler(this IApplicationBuilder app)
        {
            var loggerFactory = app.ApplicationServices.GetService(typeof(ILoggerFactory)) as ILoggerFactory;

            return app.UseExceptionHandler(HandleApiException(loggerFactory));
        }

        public static Action<IApplicationBuilder> HandleApiException(ILoggerFactory loggerFactory)
        {
            return appBuilder =>
            {
                appBuilder.Run(async context =>
                {
                    var exceptionHandlerFeature = context.Features.Get<IExceptionHandlerFeature>();

                    if (exceptionHandlerFeature != null)
                    {
                        var logger = loggerFactory.CreateLogger("Serilog Global exception logger");
                        logger.LogError(500, exceptionHandlerFeature.Error, exceptionHandlerFeature.Error.Message);
                    }

                    context.Response.StatusCode = 500;
                    await context.Response.WriteAsync("An unexpected fault happened. Try again later.");

                });
            };
        }
    }

16 You can use this by.

app.UseWebApiExceptionHandler();

Posted in ASP.NET Core | Tagged: , , , , | Leave a Comment »

Best way to protect ASP.NET Core applications from Cross Site Request Forgery(CSRF)

Posted by vivekcek on May 26, 2018

Hope you understand what is Cross Site Request Forgery(CSRF). If not just google.
ASP.NET Core has built-in support for preventing CSRF, by using a mechanism called AntiForgery Token.
The best way to validate anti forgery token in asp.net core can be done by creating a middleware.

Create a middleware as below.

using System.Threading.Tasks;
using Microsoft.AspNetCore.Antiforgery;
using Microsoft.AspNetCore.Http;

namespace Security
{

    public class ValidateAntiForgeryTokenMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly IAntiforgery _antiforgery;

        public ValidateAntiForgeryTokenMiddleware(RequestDelegate next, IAntiforgery antiforgery)
        {
            _next = next;
            _antiforgery = antiforgery;
        }

        public async Task Invoke(HttpContext context)
        {
            if (HttpMethods.IsPost(context.Request.Method))
            {
                await _antiforgery.ValidateRequestAsync(context);
            }

            await _next(context);
        }

    }
}

Now add the middleware in your startup class.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
app.UseAntiforgeryTokens();
}

In ASP.NET Core 2.0 or later, the FormTagHelper injects antiforgery tokens into HTML form elements.
So you don’t need to do anything in your view.

<form method="post">
    ...
</form>

If you want to disable antiforgery token in view do this.

<form method="post" asp-antiforgery="false">
    ...
</form>

To read about more, how to generate token from Javascript refer this link.
https://docs.microsoft.com/en-us/aspnet/core/security/anti-request-forgery?view=aspnetcore-2.0

Posted in ASP.NET Core | Leave a Comment »

EF Core DbContext in separate library

Posted by vivekcek on May 26, 2018

I was designing an ASP.NET core application with a layered architecture, and decided to keep my DbContext in a separate class library project.
This is normal in layered architecture right?. But when i tried to create migration got some strange error.

After some googling i found that we need to add below code in our library , when our DbContext and models are in separate class library.

Add below code in the library where your DbContext is, Here MyDbContext is the name of DbContext.

using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Design;

namespace DataLayer
{
    public class MyContextFactory : IDesignTimeDbContextFactory<MyDbContext>
    {
        public MyDbContext CreateDbContext(string[] args)
        {
            var builder = new DbContextOptionsBuilder<MyDbContext>();
            builder.UseSqlServer("Server=MyServer;Database=MyDatabase;Trusted_Connection=True;MultipleActiveResultSets=true");
            return new MyDbContext(builder.Options);
        }
    }
}

Posted in EfCore | Leave a Comment »