.net

memory and distributed caching in .net core

/ bonnu18 / Leave a comment

Today we shall be discussing caching. The concept of caching is rather straight-forward. The idea is to store your data on a faster secondary source, typically in memory, and not just on your primary data source, typically database. That way when your application receives requests, the data is pulled from the faster source and therefore faster response time.

In .Net Core there are two options, memory or distributed caching. Memory caching is as the name implies, in memory and it’s contained within the memory of the web server the application is running on. If your application runs on multiple web servers then distributed caching (or sticky sessions) would be a better option. Distributed caching makes your application scalable, allows session data to be shared between the web servers and does not reset when a new version is deployed. Both types of caching store values as key-value pairs. In this post I will be using Redis as my choice for distributed caching. But what if we use them both at the same time? Here’s a small POC to show how they can work together. 

I created a new .Net Core solution and selected the API template. This template comes with a default WeatherForecast controller and I used that as my skeleton to implement memory and distributed caching. I figured that the temperature is a realistic value that can be cached for a few minutes since it’s not a value that changes rapidly.

I left that untouched for now and instead created a class library to act as my business layer. In there I added a new interface and this will act as my caching service. In here I implemented the following logic; check if key is in the memory cache and if found return value. If key not found then check in distributed caching and if found return value. If key not found then look up value from primary source and save value in both memory and distributed caching. In order to connect to Redis I had to download and install the Nuget package StackExchange.Redis.

public class CacheService : ICacheService
{
private readonly IConnectionMultiplexer _muxer;
private readonly IDatabase _conn;
private readonly IMemoryCache _memCache;
public CacheService(IConnectionMultiplexer muxer, IMemoryCache memCache)
{
_muxer = muxer;
_conn = _muxer.GetDatabase();
_memCache = memCache;
}
public async Task<T> GetOrSet<T>(string key, Func<Task<T>> factory, TimeSpan cacheExpiry)
{
var value = await _memCache.GetOrCreateAsync<T>(key, entry =>
{
entry.AbsoluteExpiration = DateTime.UtcNow.Add(cacheExpiry);
return GetFromRedis(key, factory, cacheExpiry);
});
return value;
}
private async Task<T> GetFromRedis<T>(string key, Func<Task<T>> factory, TimeSpan cacheExpiry)
{
try
{
var value = await _conn.StringGetAsync(key);
if (value.HasValue)
{
try
{
return JsonConvert.DeserializeObject<T>(value);
}
catch (Exception)
{
return (T)Convert.ChangeType(value, typeof(T));
}
}
var item = await factory.Invoke();
if (item != null)
{
var serializedValue = JsonConvert.SerializeObject(item);
await _conn.StringSetAsync(key, serializedValue, cacheExpiry, When.Always, CommandFlags.None);
return item;
}
return default(T);
}
catch (Exception)
{
return default(T);
}
}
}
view raw memdistcachingone.cs hosted with ❤ by GitHub

I decided to choose an API HTTP request as my primary source instead of a database call. Sticking with the weather theme I decided to consume the Open Weather API to get that feeling of playing around with live data. Because the second parameter in the caching service endpoint is a function, I created a new weather service whose responsibility is to consume the Open Weather API. Like I said earlier this function could be a database call. In that case we would need to inject the function that retrieves the data. For completeness sake and just in case anyone would want a code snippet how to consume the Open Weather API, here’s my implementation.

public class WeatherService : IWeatherService
{
public WeatherService()
{
}
public async Task<OpenWeather> GetWeather(string cityName)
{
if (string.IsNullOrWhiteSpace(cityName))
throw new ArgumentNullException("Provide city name");
var weather = new OpenWeather();
var apiKey = "your OpenWeather API key";
using (var httpClient = new HttpClient())
{
using (var response = await httpClient.GetAsync($"https://api.openweathermap.org/data/2.5/weather?q={cityName}&appid={apiKey}&units=metric"))
{
weather = JsonConvert.DeserializeObject<OpenWeather>(await response.Content.ReadAsStringAsync());
}
}
return weather;
}
}
view raw memdistcachingtwo.cs hosted with ❤ by GitHub

I then updated the default WeatherForecast controller to use the caching service and weather service. Originally this was returning some random data and was not connected to any data source whatsoever.

[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
private readonly ILogger<WeatherForecastController> _logger;
private readonly ICacheService _cacheService;
private readonly IWeatherService _weatherService;
public WeatherForecastController(ILogger<WeatherForecastController> logger, ICacheService cacheService, IWeatherService weatherService)
{
_logger = logger;
_cacheService = cacheService;
_weatherService = weatherService;
}
[HttpGet]
public async Task<WeatherForecast> GetAsync(string city)
{
var weather = new OpenWeather();
var cacheExpiry = new TimeSpan(0, 0, 10);
weather = await _cacheService.GetOrSet<OpenWeather>(city, () => _weatherService.GetWeather(city), cacheExpiry);
return new WeatherForecast
{
Date = DateTime.Now,
TemperatureC = weather.main.temp,
Summary = weather.weather[0].description
};
}
}
view raw memdistcachingthree.cs hosted with ❤ by GitHub

The services were injected in the WeatherForecast controller using dependency injection and therefore I had to update the ConfigureServices method inside the Startup class and instantiate both services. I also added a reference to the memory and distributing caching services.

public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddMemoryCache();
services.AddSingleton<IConnectionMultiplexer>(provider => ConnectionMultiplexer.Connect("your redis connection string"));
services.AddScoped<ICacheService, CacheService>();
services.AddScoped<IWeatherService, WeatherService>();
}
view raw memdistcachingfour.cs hosted with ❤ by GitHub

Last but not least I also created some unit tests to wrap everything up nice and easy.

[TestClass]
public class CacheServiceTests
{
private CacheService _cacheService;
private Mock<IConnectionMultiplexer> _mockMuxer;
private Mock<IDatabase> _mockRedisDb;
public CacheServiceTests()
{
_mockMuxer = new Mock<IConnectionMultiplexer>();
_mockRedisDb = new Mock<IDatabase>();
}
[TestMethod]
public async Task GetOrSet_KeyFoundInMemoryCache_ReturnsValue()
{
// Arrange
var key = "TestKey";
var value = "TestValue";
var memoryCache = new MemoryCache(new MemoryCacheOptions());
memoryCache.Set(key, value);
_cacheService = new CacheService(_mockMuxer.Object, memoryCache);
// Act
var result = await _cacheService.GetOrSet<string>(key, () => Task.FromResult(value), TimeSpan.FromSeconds(30));
// Assert
Assert.IsInstanceOfType(result, typeof(string));
Assert.AreEqual(value, result);
}
}
view raw memdistcachingfive.cs hosted with ❤ by GitHub

You can find the entire solution in one of my Github repositories and feel free to test it out or make your own changes. This was just a proof of concept and can certainly do with some improvements such as storing sensitive keys or connection strings in a more secure location, or supplying two different expiry times for the memory and distributed caching. Equally the caching service could easily be put inside a “common” project and then re-used as a nuget package/artifact by different solutions.

That’s a wrap for today and I hope you enjoyed this blog post. Don’t be shy to leave any comments or get in touch if anything is unclear.

Peace out,
Bjorn

writing a fluent validation custom validator with multiple parameters

/ bonnu18 / Leave a comment

FluentValidation is a .NET framework used to validate objects and values of properties inside an object. It allows you to write different rules without much effort thanks to its out of the box rule sets validators. However different properties require different validation rules, and FluentValidation allows for custom validators to be written, which again I think it’s great. Having said that, I recently needed to write a validation rule set for a property that depended also on the value of another property (in the same object).

Here’s what I came up with in a scenario where a DTO is received in an API endpoint used when users purchase Playstation games.


public class PurchasePlaystationGameDto
{
public int ID { get; set; }
public int UserID { get; set; }
public decimal Price { get; set; }
public bool PlaystationPlusMember { get; set; }
}
public class PurchasePlaystationGameDtoValidator : AbstractValidator
{
public PurchasePlaystationGameDtoValidator()
{
RuleFor(x => x.ID).NotNull().GreaterThan(0).WithMessage("ID must be greater than 0.");
RuleFor(x => x.UserID).NotNull().GreaterThan(0).WithMessage("UserID must be greater than 0.");
RuleFor(x => x.Price).Must(BeGreaterThanZeroForNonMembers).WithMessage("Price must be greater than 0");
}
private bool BeGreaterThanZeroForNonMembers(PurchasePlaystationGameDto dtoInstance, decimal price)
{
// game is free for members 🙂
if (!dtoInstance.PlaystationPlusMember & price <= 0)
{
return false;
}
return true;
}
}

view raw

customfluentvalidator.cs

hosted with ❤ by GitHub

Simple and straight forward, if the user is a Playstation plus member then the price can be zero as the member is eligible for a free game.

That’s a wrap, until next post.
Bjorn

mocking a method to return different results

/ bonnu18 / Leave a comment

This week I was working on a piece of code that process batches and it had to be unit tested. Let’s think of this scenario. We have a service endpoint that creates a batch order (and a record in the database), and then for each item in that batch order it will create a product order (and again, a record in the database). If one of the product order fails, we need to revert the process, and delete all product orders together with the batch order.

Implementing it wasn’t a problem but when it came to unit testing the behaviour in the case when one product order fails, I had to think twice about it. Thankfully I discovered the SetupSequence method in the testing framework that I use (xUnit). What this method does is basically saying that when a method is called, I want the first value to be something, and the second value to be something else, in my case zero. I then decided to assert that not only the return would be zero (because the whole batch failed) but also that the method that reverts the transaction was called. Check out my implementation below;


[Fact]
public async void TheMethodImMocking_Returns_Zero_If_One_Of_The_Records_Not_Inserted()
{
// Arrange
Mock _mockDataLayerService = new Mock;
_mockDataLayerService.SetupSequence(x => x.ProcessIndividualOrder(It.IsAny())).ReturnsAsync(9999).ReturnsAsync(0);
// Act
var batchProcessingService = new BatchProcessingService(_mockDataLayerService.Object);
var result = batchProcessingService.ProcessBatchOrder()
// Assert
Assert.IsType(result);
Assert.Equal(0, result);
_mockDataLayerService.Verify(m => m.ReverseOrderProcessing(It.IsAny()), Times.Exactly(1));
}

view raw

mockdifferentresults.cs

hosted with ❤ by GitHub

The only catch when using the SetupSequence method is to know how many times the testable method will be called because if the return values, and the amount of calls do not match, an exception will be throw. Again, in my case I knew that I will be calling the method ProcessIndividualOrder twice so I setup the method to return a value twice.

I hope you’ll find this useful the same way I did, and happy unit testing 🙂

Bjorn

implementing a cascading dropdown list with MVC and Jquery

/ bonnu18 / Leave a comment

By definition (my own) a cascading dropdown list(s) is a set of dropdown list(s) dependent on their parent dropdown list. In this blog post we’ll see how this can be achieved in ASP.NET’s MVC pattern, with the help of some client-side scripting too. For simplicity I will be using the standard template that Visual Studio creates for you when you create a new project. Also, I uploaded the source code to this GitHub location.

All right so let’s say that we’re going to have three dropdown lists, and through these we will be able to drill down from continent, to countries, to cities. All the data in this project is hard coded but it can easily be retrieved from a database or some other source. So first thing we’ll do is update the Index method in the HomeController to make it look like the following;


public ActionResult Index()
{
continents = new List();
continents.Add(new SelectListItem { Text = "Select continent", Value = "0" });
continents.Add(new SelectListItem { Text = "Europe", Value = "1" });
continents.Add(new SelectListItem { Text = "North America", Value = "2" });
continents.Add(new SelectListItem { Text = "South America", Value = "3" });
ViewData["continents"] = continents;
countries = new List();
countries.Add(new SelectListItem { Text = "Select country", Value = "0" });
ViewData["countries"] = countries;
cities = new List();
cities.Add(new SelectListItem { Text = "Select city", Value = "0" });
ViewData["cities"] = cities;
return View();
}

view raw

cascadingdropdownone.cs

hosted with ❤ by GitHub

So we created three lists of type SelectListItem and filling them with some data, and then passing them to our view using the ViewData. These lists can also be bound to a model and then pass the model to the view. I chose the ViewData option for simplicity. Within the view we need to create three dropdown lists and make a reference to the ViewData.


<div class="row">
<div class="col-md-4">
@Html.DropDownList("continents", ViewData["continents"] as List)
</div>
<div class="col-md-4">
@Html.DropDownList("countries", ViewData["countries"] as List)
</div>
<div class="col-md-4">
@Html.DropDownList("cities", ViewData["cities"] as List)
</div>
</div>

view raw

cascadingdropdowntwo.html

hosted with ❤ by GitHub

Now we got three dropdown lists with some data in them. For the next part we need to add some client-side scripting so we can load more data dynamically. Using Jquery we will make an AJAX call and get data according to what the user selected.


$("#continents").change(function () {
$("#countries").empty();
$.ajax({
type: 'POST',
url: '/Home/GetCountries',
dataType: 'json',
data: { selectedValue: $("#continents").val() },
success: function (areas) {
$.each(areas, function (i, area) {
$("#countries").append('' + area.Text + '');
});
},
error: function (ex) {
alert('Failed to retrieve countries.' + ex);
}
});
return false;
});
$("#countries").change(function () {
$("#cities").empty();
$.ajax({
type: 'POST',
url: '/Home/GetCities',
dataType: 'json',
data: { selectedValue: $("#countries").val() },
success: function (cases) {
$.each(cases, function (i, ucase) {
$("#cities").append('' + ucase.Text + '');
});
},
error: function (ex) {
alert('Failed to retrieve cities.' + ex);
}
});
return false;
});

view raw

cascadingdropdownthree.js

hosted with ❤ by GitHub

Back to the HomeController we are going to create two entry points and do our logic in there.


public JsonResult GetCountries(string selectedValue)
{
countries = new List();
countries.Add(new SelectListItem { Text = "Select country", Value = "0" });
switch (selectedValue)
{
case "1":
countries.Add(new SelectListItem { Text = "United Kingdom", Value = "1" });
countries.Add(new SelectListItem { Text = "Italy", Value = "2" });
countries.Add(new SelectListItem { Text = "Greece", Value = "3" });
break;
case "2":
// more countries
break;
case "3":
// more countries
break;
}
return Json(new SelectList(countries, "Value", "Text"));
}
public JsonResult GetCities(string selectedValue)
{
cities = new List();
cities.Add(new SelectListItem { Text = "Select city", Value = "0" });
switch (selectedValue)
{
case "1":
cities.Add(new SelectListItem { Text = "Glasgow", Value = "1" });
cities.Add(new SelectListItem { Text = "London", Value = "2" });
cities.Add(new SelectListItem { Text = "Edinburgh", Value = "3" });
break;
case "2":
// more cities
break;
case "3":
// more cities
break;
case "4":
// more cities
break;
case "5":
// more cities
break;
// more cases
}
return Json(new SelectList(cities, "Value", "Text"));
}

view raw

cascadingdropdownfour.cs

hosted with ❤ by GitHub

And that should be it. Feel free to share your thoughts if you have any suggestions. Until the next post.

Bjorn

basic implementation of AWS SNS topic using c#

/ bonnu18 / Leave a comment

Cloud computing, or simply the cloud, has been quite the buzz word in recent years together with cloud service providers like Microsoft Azure, Google Cloud Platform and also – drum roll please – Amazon Web Services, also known as AWS. AWS has been around more than you think, since 2006 if we had to be exact, and in the developers vocabulary it’s really starting to become trendier.

One of the features AWS offers is Simple Notification Service (SNS). The idea behind it is a notification system to push new notifications to all users, or devices, subscribed to it. An SNS Topic is what users subscribe to and is the channel that will enable us to get in touch with them. In this blog post, however, we are going to assume that a topic has already been created and that we know the value of its ARN.

Now let’s focus on how we can send a request to that topic to trigger a new push notification. First thing we need to do is download and install two NuGet packages in our Visual Studio solution.

  • AWSSDK.Core
  • AWSSDK.SimpleNotificationService

Create a new C# class and paste the following code.


using Amazon;
using Amazon.Runtime;
using Amazon.SimpleNotificationService;
using Amazon.SimpleNotificationService.Model;
using System.Net;
class SnsEvents
{
const string AccessKey = "ABCDEFGHIJKLMNOPQRST";
const string SecretKey = "AbcdeFghiJKlmopqrS39VwXY0Za/1cdEF5I9aC";
const string SnsArn = "arn:aws:sns:eu-west-1:004673491943:TheNameOfTheSnsTopic";
static readonly RegionEndpoint Region = RegionEndpoint.EUWest1; //Same region as the ARN above
private static bool publishSnsRequest(string _message)
{
bool success = false;
AWSCredentials credentials;
credentials = new BasicAWSCredentials(AccessKey, SecretKey);
var client = new AmazonSimpleNotificationServiceClient(credentials, Region);
var request = new PublishRequest(SnsArn, _message);
var response = new PublishResponse();
response = client.Publish(request);
string requestId = response.ResponseMetadata.RequestId;
if (response.HttpStatusCode == HttpStatusCode.OK) {
success = true;
}
return success;
}
}

view raw

awssnstopic.cs

hosted with ❤ by GitHub

Analysing the code we can see that I declared four constant variables which will be used to connect to the SNS Topic. The values of these variables should be provided to you by the person maintaining the SNS. Also, notice the value of the region variable. Same region as specified in SNS’s ARN. Then, using the AWSCredentials object, we create a new credentials instance by supplying the access and secret key in the constructor’s parameters. A new AWS SNS client instance is created and the message that we want to send is published. Afterwards we check the response to make sure that our request was processed successfully.

A quick and straight forward post with a basic implementation of how to publish an SNS Topic from a .NET solution. There are much more options one can implement but this should be enough to get you started.

Until next post,
Bjorn