dotnetcore

instance of entity type cannot be tracked when unit testing ef core

/ bonnu18 / Leave a comment

Recently I was unit testing a service implementation that handles manipulation of data to and from an API, and I came across this peculiar exception.

System.InvalidOperationException : The instance of entity type 'tblExcludedSellers' cannot be tracked because another instance with the same key value for {'SellerId', 'Username'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached. Consider using 'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the conflicting key values.

Strangely enough this exception was being thrown only when unit testing and stepping over inside the service endpoint and not during runtime. This happened to me at work but I managed to reproduce the error in the scenario that I have mentioned in my previous post. We have a system that alerts users when PlayStation 5 consoles are back in stock from a list of defined sellers. This system has an API along with a backend service for data transfers. One thing I’d like to point out is that the database is designed the other way round and what I mean by that is that the user will receive a notification from all sellers unless s/he selects the sellers s/he wants to exclude. Yes I know it’s bad database design but this is what I was working with and it wasn’t possible for me to change this. Let’s have a look at our service endpoint which is ran whenever the user would like to update his/her list of sellers.

public class UserSettingsService : IUserSettingsService
{
private readonly StockNotificationContext _dbContext;
public UserSettingsService(StockNotificationContext dbContext)
{
_dbContext = dbContext;
}
public async Task UpdateUserSellerPreferences(string username, IEnumerable<int> sellerIds)
{
var allSellerIds = await _dbContext.tblSellers.Select(x => x.Id).ToListAsync();
var currentExclusions = await _dbContext.tblExcludedSellers.Where(x => x.Username == username).Select(x => x.SellerId).ToListAsync();
// Determine insertions and removals based on provided ids and what's currently in db
var correctExclusions = allSellerIds.Except(sellerIds);
var removals = currentExclusions.Except(correctExclusions)
.Select(x => new tblExcludedSellers
{
Username = username,
SellerId = x,
});
var insertions = correctExclusions.Except(currentExclusions)
.Select(x => new tblExcludedSellers
{
Username = username,
SellerId = x,
});
_dbContext.tblExcludedSellers.AddRange(insertions);
_dbContext.tblExcludedSellers.RemoveRange(removals);
await _dbContext.SaveChangesAsync();
}
view raw efcoretrackedissueone.cs hosted with ❤ by GitHub

The flow can be summarised in the following steps;

  1. Get all the seller IDs.
  2. Get the current user’s excluded sellers.
  3. Compare the list of sellers from Step 1 with the list of IDs from the method parameter. That represents a list of IDs that the user wants to receive notifications from (think along the lines of a user selecting sellers from checkboxes).
  4. Sellers that need to be added or deleted in the database tables are determined by comparing the list of IDs from step 2 and step 3, and then in one transaction the database is updated.

Naturally I wanted to unit test that and this was my first attempt (the one that was giving me the exception).

public class UserSettingsServiceTests
{
private readonly DbContextOptions<StockNotificationContext> _options;
private readonly StockNotificationContext _dbContext;
private readonly UserSettingsService _userSettingsService;
private const string _username = "UnitTestUsername";
public UserSettingsServiceTests()
{
_options = new DbContextOptionsBuilder<StockNotificationContext>().UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString()).Options;
_dbContext = new StockNotificationContext(_options);
_userSettingsService = new UserSettingsService(_dbContext);
}
[Fact]
public async Task UpdateUserSellerPreferences_Updates_User_Settings()
{
// Arrange
await _dbContext.Database.EnsureDeletedAsync();
var listOfSellers = new List<tblSellers>();
listOfSellers.Add(new tblSellersBuilder().WithId(1).WithName("Amazon").WithUrl("https://www.amazon.co.uk&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(2).WithName("Ebay").WithUrl("https://www.ebay.co.uk&quot;).Build());
// other sellers added here
_dbContext.tblSellers.AddRange(listOfSellers);
var listOfExclSellers = new List<tblExcludedSellers>();
listOfExclSellers.Add(new tblExcludedSellersBuilder().WithUsername(_username).WithSellerId(5).Build());
listOfExclSellers.Add(new tblExcludedSellersBuilder().WithUsername(_username).WithSellerId(6).Build());
_dbContext.tblExcludedSellers.AddRange(listOfExclSellers);
await _dbContext.SaveChangesAsync();
var newListOfSellers = new List<int>() { 1, 2, 3, 5 };
// Act
var task = _userSettingsService.UpdateUserSellerPreferences(_username, newListOfSellers);
await task;
// Assert
var updatedList = _dbContext.tblExcludedSellers.Where(x => x.Username == _username).Select(x => x.SellerId).ToList();
Assert.Equal(2, updatedList.Count);
Assert.Contains(4, updatedList);
Assert.Contains(6, updatedList);
}
}
view raw efcoretrackedissuetwo.cs hosted with ❤ by GitHub

This would normally work for me but in this case it didn’t and at first I couldn’t understand why. I googled it up and this is how I understood it. When setting up the unit test I create an in-memory database and add test data to it. When the test data is added it is being tracked (especially since it’s added with the .AsNoTracking() method) and then that same database instance is injected. When the unit test processor attempts to remove or add excluded sellers, the EF core tracker throws the exception as the data is “attached” and is already being tracked. I would like to point out that in my case the database table, tblExcludedSellers, didn’t have a primary key, but a composite key, and didn’t have an identity column (an auto-increment default value). This was highlighted as a potential issue in this thread. If it’s of any help here’s my database table key binding found inside the context’s model creating method.

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<tblExcludedSellers>(entity =>
{
entity.HasKey(e => new { e.SellerId, e.Username });
});
}
view raw efcoretrackedissuefour.cs hosted with ❤ by GitHub

I solved this by creating a separate in-memory database but using the same database context options and injecting different instance, again same database context options.

public class UserSettingsServiceTests
{
private readonly DbContextOptions<StockNotificationContext> _options;
private readonly StockNotificationContext _dbContext;
private readonly UserSettingsService _userSettingsService;
private const string _username = "UnitTestUsername";
public UserSettingsServiceTests()
{
_options = new DbContextOptionsBuilder<StockNotificationContext>().UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString()).Options;
_dbContext = new StockNotificationContext(_options);
_userSettingsService = new UserSettingsService(_dbContext);
}
[Fact]
public async Task UpdateUserSellerPreferences_Updates_User_Settings()
{
// Arrange
await _dbContext.Database.EnsureDeletedAsync();
using (var seedingContext = new StockNotificationContext(_options))
{
var listOfSellers = new List<tblSellers>();
listOfSellers.Add(new tblSellersBuilder().WithId(1).WithName("Amazon").WithUrl("https://www.amazon.co.uk&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(2).WithName("Ebay").WithUrl("https://www.ebay.co.uk&quot;).Build());
// other sellers added here
seedingContext.tblSellers.AddRange(listOfSellers);
var listOfExclSellers = new List<tblExcludedSellers>();
listOfExclSellers.Add(new tblExcludedSellersBuilder().WithUsername(_username).WithSellerId(5).Build());
listOfExclSellers.Add(new tblExcludedSellersBuilder().WithUsername(_username).WithSellerId(6).Build());
seedingContext.tblExcludedSellers.AddRange(listOfExclSellers);
await seedingContext.SaveChangesAsync();
}
var newListOfSellers = new List<int>() { 1, 2, 3, 5 };
// Act
var task = _userSettingsService.UpdateUserSellerPreferences(_username, newListOfSellers);
await task;
// Assert
var updatedList = _dbContext.tblExcludedSellers.Where(x => x.Username == _username).Select(x => x.SellerId).ToList();
Assert.Equal(2, updatedList.Count);
Assert.Contains(4, updatedList);
Assert.Contains(6, updatedList);
}
}
view raw efcoretrackedissuethree.cs hosted with ❤ by GitHub

What also worked for others was detaching the entities after adding them, as pointed out in this Stack Overflow thread. It seems that this exception is some what common but from what I understood by going through different threads is that this exception can be thrown for different reason (and not specifically for the scenario I created in this post). Having said that the above might work for you and for that reason I uploaded my solution to GitHub for anyone who like to fiddle around with the code. Thanks a lot for reading and feel free to comment below if you feel I might have missed something out.

Until next post,
Bjorn

using entity framework core in-memory database provider to create a database on the fly

/ bonnu18 / 1 Comment

Were you ever in a situation where you needed a quick, handy database but didn’t want to spend a lot of time connecting everything up? Maybe you just need to test a small database table but importing the entire schema takes ages? Well, that was my case and after managing to get one up and running, I wanted to share with you how I got there. The technologies I’m currently working on are .NET Core v3.1 and using Entity Framework Core v5.0 (Nuget package Microsoft.EntityFrameworkCore v5.0.1). Additionally I also had to install the Nuget package Microsoft.EntityFrameworkCore.InMemory.

The scenario in this case is the following; imagine we have a system that notifies users when a PlayStation 5 is in stock at a seller’s store (it would be a million dollar idea right now 😀). There’s a defined list of PS5 seller (Amazon, Ebay, etc) and a user can select to receive stock notifications from this list of sellers. For the sake of this blogpost there’s no front end technologies but just an API. In fact in this case I create a new Visual Studio solution and selected the ASP.NET Core Web Application template and left the standard API option.

Next thing I did was add a new class library project and this will serve as the data layer. I then added a new model with properties to mimic a database table, and how it would be created by the EF model creation process. Similarly I created another class to act as the database context. These implementation can be found below.

public partial class tblSellers
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public string Name { get; set; }
public string Url { get; set; }
}
view raw efcoreinmemoryone.cs hosted with ❤ by GitHub

public partial class StockNotificationContext : DbContext
{
public StockNotificationContext()
{
}
public StockNotificationContext(DbContextOptions<StockNotificationContext> options) : base(options)
{
}
public virtual DbSet<tblSellers> tblSellers { get; set; }
}
view raw efcoreinmemorytwo.cs hosted with ❤ by GitHub

Then I added another class to generate and build the data. All the individual property methods and randomisation of data aren’t really necessary but it’s a practice that we follow at work and I kind of picked up this good habit. I find them useful when working on unit tests as you can play around with data to satisfy test criteria.

public class tblSellersBuilder
{
private readonly tblSellers _tblSellers;
private readonly Random _random;
public tblSellersBuilder(Random random = null)
{
_random = random ?? new Random();
_tblSellers = new tblSellers
{
Id = _random.Next(),
Name = _random.Next().ToString(),
Url = _random.Next().ToString(),
};
}
public tblSellers Build()
{
return _tblSellers;
}
public tblSellersBuilder WithId(int id)
{
_tblSellers.Id = id;
return this;
}
public tblSellersBuilder WithName(string name)
{
_tblSellers.Name = name;
return this;
}
public tblSellersBuilder WithUrl(string url)
{
_tblSellers.Url = url;
return this;
}
public static void Initialize(StockNotificationContext stockNotificationContext)
{
var listOfSellers = new List<tblSellers>();
listOfSellers.Add(new tblSellersBuilder().WithId(1).WithName("Amazon").WithUrl("https://www.amazon.co.uk&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(2).WithName("Ebay").WithUrl("https://www.ebay.co.uk&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(3).WithName("Currys PC World").WithUrl("https://www.currys.co.uk&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(4).WithName("Argos").WithUrl("https://www.argos.co.uk&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(5).WithName("Smyths").WithUrl("https://www.smythstoys.com&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(6).WithName("Target").WithUrl("https://www.target.com&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(7).WithName("Best Buy").WithUrl("https://www.bestbuy.com&quot;).Build());
listOfSellers.Add(new tblSellersBuilder().WithId(8).WithName("Walmart").WithUrl("https://www.walmart.com&quot;).Build());
stockNotificationContext.tblSellers.AddRange(listOfSellers);
stockNotificationContext.SaveChanges();
}
}
view raw efcoreinmemorythree.cs hosted with ❤ by GitHub

I then registered the database and context in the Startup.cs file, ConfigureServices method. After that I added a call to the data generator class in the Configure service so that when my application loads it would have some data to work with.

public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddDbContext<StockNotificationContext>(options => options.UseInMemoryDatabase(databaseName: "StockNotification"));
services.AddScoped<IUserSettingsService, UserSettingsService>();
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
// this should be here when you create the solution
app.UseHttpsRedirection();
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
using (var serviceScope = app.ApplicationServices.CreateScope())
{
var dbContext = serviceScope.ServiceProvider.GetService<StockNotificationContext>();
tblSellersBuilder.Initialize(dbContext);
}
}
view raw efcoreinmemoryfour.cs hosted with ❤ by GitHub

I created another class library project to act as the business layer. In here I added a new service and a service endpoint to get a list of sellers (the same that I generate on application start up). This service is the same one registered in the Startup.cs above.  Finally to wrap it up, I created a new controller in the original API project and referenced the service I have just created to be able to provide the user with the list of sellers. Implementations below. I also created an interface for the service and a DTO to return a list of that instead of the database model but omitting them here to keep it short(ish).

public class UserSettingsService : IUserSettingsService
{
private readonly StockNotificationContext _dbContext;
public UserSettingsService(StockNotificationContext dbContext)
{
_dbContext = dbContext;
}
public async Task<IEnumerable<Seller>> GetSellers()
{
return await _dbContext.tblSellers
.Select(x => new Seller
{
Id = x.Id,
Name = x.Name,
Url = x.Url
})
.ToListAsync();
}
}
view raw efcoreinmemoryfive.cs hosted with ❤ by GitHub

[ApiController]
[Route("[controller]")]
public class SettingsController : ControllerBase
{
private readonly IUserSettingsService _userSettingsService;
public SettingsController(IUserSettingsService userSettingsService)
{
_userSettingsService = userSettingsService;
}
[HttpGet]
public ObjectResult GetSellers()
{
return Ok(_userSettingsService.GetSellers().Result);
}
}
view raw efcoreinmemorysix.cs hosted with ❤ by GitHub

And that should be enough to have a database working in memory during runtime! At the beginning of this post I came up with this stock notification scenario and that is tied to another post which I will be writing in the coming days. I will discuss an EF related issue I came across and how to fix it. I will also be putting a link to the entire solution on GitHub. If this post helps and would like to donate a PS5 please get in touch 😀

Until next post,
Bjorn

unit testing an iformfile that is converted into system.drawing.image in c#

/ bonnu18 / Leave a comment

In this blog post I’m going to be covering a very specific issue. Let’s imagine the following scenario; we have a RESTful API running on .NET Core that uses a series of classes, disguised as services, (again running on .NET Core) as it’s business layer. The RESTful API receives HTTP requests, said requests are processed by the services, there’s some data manipulation happening, and then a result is returned back. The class library project would have a unit test project that tests the functionality inside it. So far, I’d like to think, is rather clear and quite a standard approach too. The following is our MVC controller that receives requests related to images. The image service is injection via Dependency Injection and then the method SaveImage() is called.

[Route("api/[controller]")]
[ApiController]
public class ImagesController : ControllerBase
{
private readonly IImageService _imageService;
public ImagesController(IImageService imageService)
{
_imageService = imageService;
}
// POST api/images
[HttpPost]
public async Task<IActionResult> PostAsync([FromForm] ImageDetailsDto imageDetails)
{
var requestImage = Request.Form.Files.FirstOrDefault();
var result = await _imageService.SaveImage(imageDetails.UserId, requestImage);
// save user details in some other service
return Ok(result);
}
}
view raw unittestingiformfileone.cs hosted with ❤ by GitHub

The image service (which inherits from an interface) grabs the IFormFile object, converts it to an Image (from the Nuget Package System.Drawing), checks the width and height, and saves accordingly. That implementation can be found in the following snippet.

public class ImageService : IImageService
{
public async Task<string> SaveImage(int userId, IFormFile uploadedImage)
{
// convert IFormFile to Image and validate
using (var image = Image.FromStream(uploadedImage.OpenReadStream()))
{
if (image.Width > 640 || image.Height > 480)
{
// do some resizing and then save image
}
else
{
// save original image for user
}
}
return "image saved";
}
}
view raw unittestingiformfiletwo.cs hosted with ❤ by GitHub

Eventually we’re going to want to unit test this interface and this is where I ran into an issue. I was able to create a mock, so to speak, IFormFile and mimic the behaviour of an image as one of the parameters of the method SaveImage, but as soon as I tried to convert that mocked IFormFile into an Image my program threw an exception. From the way I understood it, the IFormFile is essentially a Stream. In an actual HTTP request that Stream represents an image (with all it’s graphical data compressed in that Stream) and is compatible with the object Image (Sytem.Drawing) but when I created a random Stream for my unit test, that Stream is lacking graphical data and therefore cannot be converted to an Image. I then started digging on Google, and StackOverflow, and thanks to this guy’s blog post I came up with a solution. Create an actual graphical image, convert it into a Stream and then inject that in the test, as you can see below.

public class ImageServiceTests
{
private readonly IImageService _imageService;
private readonly int _userId = 1234;
public ImageServiceTests()
{
_imageService = new ImageService();
}
[Fact]
public async Task Service_Saves_Image()
{
// Arrange
var expected = "image saved";
var imageStream = new MemoryStream(GenerateImageByteArray());
var image = new FormFile(imageStream, 0, imageStream.Length, "UnitTest", "UnitTest.jpg")
{
Headers = new HeaderDictionary(),
ContentType = "image/jpeg"
};
// Act
var result = await _imageService.SaveImage(_userId, image);
// Assert
Assert.Equal(expected, result);
}
private byte[] GenerateImageByteArray(int width = 50, int height = 50)
{
Bitmap bitmapImage = new Bitmap(width, height);
Graphics imageData = Graphics.FromImage(bitmapImage);
imageData.DrawLine(new Pen(Color.Blue), 0, 0, width, height);
MemoryStream memoryStream = new MemoryStream();
byte[] byteArray;
using (memoryStream)
{
bitmapImage.Save(memoryStream, ImageFormat.Jpeg);
byteArray = memoryStream.ToArray();
}
return byteArray;
}
}
view raw unittestingiformfilethree.cs hosted with ❤ by GitHub

I also took the liberty of uploading the solution to my github in case anyone would want to have a better look at it. I hope this post has been helpful to you and thanks for reading 🙂

Until next post,
Bjorn