Hi, John,

ImageGen should work perfectly in this situation and there are sites with vastly more images being processed. I wonder, do you have the &nocache=true parameter set? You shouldn't except perhaps for some testing.

What happens is the 60 images on your homepage will make 60 requests to imagegen.ashx. ImageGen will resize each according to your height/width and other parameters. Those resized images will be saved to the server's drive, the index.xml file(s) will be updated, and the resized images will be sent to the website visitor. Resizing can be a cpu and memory intensive task, especially if the original images are very large. But it only happens once for each unique set of height/width and other parameters.

Once ImageGen's cache is primed, any future requests will find the cached file, update the associated index.xml file, and send the already-cached image to the website visitor. This is less cpu-intensive but like all website activity opening 60 http requests and sending the data down the pipe may be a load on the server for a moment, depending on the server's robustness. ImageGen is adding very little to this overhead than if you'd requested the images from the filesystem directly, but 60 nearly simultaneous requests for each website visitor can add up even for static resources.

Or, if the website visitor has already downloaded the images from a previous visit to the site, ImageGen will respond with a 304 Not Modified response to the requests and bandwidth will be reduced to nearly zero for each request. Apart from the momentary burst of 60 requests and 60 304 responses. 

Finally, if you are using ImageGen Professional, you can set a CachingTimeSpan value for a class of images and the visitor's browser won't even request the image again from your server for that time period, totally eliminating the 60 requests and 304 responses once the visitor gets the images initially. No activity on the server means zero load on the server. You can always try the ImageGen Professional features by running on localhost or any *.local domain before purchasing a key for your live domain.

I hope this explanation helps, and do be sure you're not using the &nocache=true parameter as that will force ImageGen to recreate the image (ignoring any previously-cached images) with every request.

cheers,
doug. 

Hi Doug,

Definately don't have a &nocache=true parameter set

I changed the URL paths from ImageGen.ashx to the direct generated cache image and there was no CPU movement so it wasn't the server / bandwidth. 

After a little more investigation a handful of URL's had the image path and Contrain=True and nothing else. After looking into the index.xml file all the cached images had height/width values so I assume it wasn't caching those images (with just Constain=True) but still serving them. Adding heights/widths to those calls dropped the CPU hit by about half, down to roughly 10% with 40 images. Still this seems high.

FYI, it's ImageGenPro version 2.5.7.27945

Also, if I add MaxHeight and MaxWidth to the request they don't seem to be stored in the index.xml. In fact, adding MaxHeight or MaxWidth doesn't seem to do anything. I know ImageGenPro is working because if I add Class=X it renders the class settings. Is it not valid to simply have MaxHeight and/or MaxWidth ?

Hi, John,

(I've been very sick and out of the office for many days, sorry for the slow response)

Thanks for the info, that's helpful. 

The index.xml file will always have every key parameter in it, not only those you specify. That way, if you request a different set of parameters but the result would be the same image ImageGen doesn't need to re-create an already-existing image. Constrain and maxWidth and maxHeight aren't part of the index.xml since they don't define the resulting image (an image with maxWidth=50 will produce an image with a width of 50, and the resulting width is saved in the index.xml... but you could get the same result by requesting the image with width=50, for instance, amongst other potential combinations of parameters).

An image that doesn't need any process is sent to the visitor directly and isn't resized or cached. 

I wonder if there's some poorly performing code in some of these routines to determine the resulting parameters, or possibly in opening files if they aren't being resized and therefore aren't cached. It's worth a look.

cheers,
doug. 

No problem Doug, understandable.

One thought I had was looking up the cached image path myself (parsing index.xml) and caching that path in my application, saving it from hitting ImageGen for each image request. Is there any API within ImageGen that'll allow this? If I use ImageGen.ImageGenQueryStringParser.ImageGen ?

Cheers, feel better.

I'm seeing similar load problems, although not yet worked out if there's a similar cause - we've just sent an email marketing campaign out, and the affected site is now serving ~20-40 concurrent ImageGen requests, and melting down with >95% CPU load.

We are using ImageGen Pro in our ASP.Net appllication hosted in our server with 16GB memory and 2 Quadcore Xeon processors at 2.5GHz. Our site has to render between 60 - 80 images per minute. Average size of the images is 110KB. We are using the bellow classes, and still we get high usage of the resources. Sometimes it goes as far as 100% of the CPU. Is there a way to improve this situation please.

   <Class Name="Thumbnail" OverridesQueryString="true">
        <AllowUpsizing>False</AllowUpsizing>
        <Width>269</Width>
        <Height>196</Height>
        <Transparent>False</Transparent>
        <Pad>True</Pad>
        <CachingTimeSpan>86400</CachingTimeSpan>
    </Class>
    <Class Name="Main" OverridesQueryString="true">
        <AllowUpsizing>False</AllowUpsizing>
        <Width>439</Width>
        <Height>293</Height>
        <Transparent>False</Transparent>
        <Pad>True</Pad>
        <CachingTimeSpan>86400</CachingTimeSpan>
    </Class>
    <Class Name="Summary" OverridesQueryString="true">
        <AllowUpsizing>False</AllowUpsizing>
        <Width>182</Width>
        <Height>137</Height>
        <Transparent>False</Transparent>
        <Pad>True</Pad>
        <CachingTimeSpan>86400</CachingTimeSpan>
    </Class>
    <Class Name="Gallery" OverridesQueryString="true">
        <AllowUpsizing>False</AllowUpsizing>
        <Width>187</Width>
        <Height>142</Height>
        <Transparent>False</Transparent>
        <Pad>True</Pad>
        <CachingTimeSpan>86400</CachingTimeSpan>
    </Class>

Hi Byron,

I ended up wrting a very basic method (well, few methods) that would look at the ImageGen XML files and cache the physical image paths of the generated ImageGen images so image requests aren't constantly hitting ImageGen.ashx. They're customised for our needs but you can probably modify them for yours. Essentially, every image request calls RenderImageGenUrl and from there it calls another method (GetDirectImageUrl) to check the cache and then (if neccesary) another method to lookup the XML file (RetrieveGeneratedImageUrl).

public static string RenderImageGenUrl(ImageGenSettingsEntity settings, bool encodeAmpersands = true, bool useCache = true)
{
   string imageGenUrl, relativeImageUrl;
 string amp;

 imageGenUrl = System.Configuration.ConfigurationManager.AppSettings["ImageGenUrl"];
   amp = (encodeAmpersands ? "&" : "&");

   if (String.IsNullOrEmpty(imageGenUrl)) throw new NullReferenceException("AppSettings missing setting 'ImageGenUrl'");
 if (!imageGenUrl.Contains("?")) imageGenUrl = String.Concat(imageGenUrl, "?");

  if (String.IsNullOrEmpty(settings.ImageUrl))
  {
     return String.Empty;
  }

   // add the image
  relativeImageUrl =
        settings.ImageUrl.Replace(
            String.Concat("http://", new Uri(imageGenUrl).Host),
          String.Empty
      );
    imageGenUrl =
     String.Concat(
            imageGenUrl,
          "Image=",
         System.Web.HttpUtility.UrlEncode(relativeImageUrl)
        );

  // add remaining properties
   if (settings.Height > 0) imageGenUrl = String.Concat(imageGenUrl, amp, "Height=", settings.Height);
    if (settings.Width > 0) imageGenUrl = String.Concat(imageGenUrl, amp, "Width=", settings.Width);
   if (settings.Constrain && (settings.Height > 0 || settings.Width > 0)) imageGenUrl = String.Concat(imageGenUrl, amp, "Constrain=", true.ToString());
    if (!String.IsNullOrEmpty(settings.Class)) imageGenUrl = String.Concat(imageGenUrl, amp, "Class=", settings.Class);
   if (!settings.AllowUpsizing) imageGenUrl = String.Concat(imageGenUrl, amp, "AllowUpSizing=", false.ToString());
   if (settings.MaxHeight > 0)
    {
     imageGenUrl = String.Concat(imageGenUrl, amp, "Height=", settings.MaxHeight);
     if (settings.AllowUpsizing) imageGenUrl = String.Concat(imageGenUrl, amp, "AllowUpSizing=", false.ToString());
    }
 if (settings.MaxWidth > 0)
 {
     imageGenUrl = String.Concat(imageGenUrl, amp, "MaxWidth=", settings.MaxWidth);
        if (settings.MaxHeight <= 0 && settings.AllowUpsizing) imageGenUrl = String.Concat(imageGenUrl, amp, "AllowUpSizing=", false.ToString());
  }

   // attempt to retrieve the path from cache
    return
        useCache
          ? GetDirectImageUrl(settings, imageGenUrl, relativeImageUrl)
          : imageGenUrl;
}
     
public string GetDirectImageUrl(ImageGenSettingsEntity imageGenSettings, string generatedUrl, string relativeImageUrl)
{
   _logger.Debug("GetDirectImageUrl");

 var key = String.Format(KEY_ASSET_URL, generatedUrl);
 var cachedUrl = base.CacheHandler.Get<CachedImageGenUrlEntity>(key);

  if (cachedUrl == null || cachedUrl.AttemptOverride)
   {
     _logger.Debug(String.Format("Retrieve from service {0}", relativeImageUrl));

        var url = base.Service.RetrieveGeneratedImageUrl(imageGenSettings, relativeImageUrl);

       if (cachedUrl == null)
        {
         cachedUrl = new CachedImageGenUrlEntity(generatedUrl);
        }
     else
      {
         cachedUrl.Url = generatedUrl;
         cachedUrl.AttemptOverride = false;
        }

       if (!String.IsNullOrEmpty(url))
       {
         cachedUrl.Url = url;
      }

       base.CacheHandler.Set(key, cachedUrl, base.CacheDuration);
    }

   return cachedUrl.Url;
}

public string RetrieveGeneratedImageUrl(ImageGenSettingsEntity imageGenSettings, string relativeImageUrl)
{
  _logger.Debug("RetrieveGeneratedImageUrl");

 if (String.IsNullOrEmpty(relativeImageUrl)) return null;

    decimal ratio;
    int resizedHeight, resizedWidth;

    var isExternalUrl = (relativeImageUrl.StartsWith("http://") || relativeImageUrl.StartsWith("https://"));
  var requestedUri = OperationContext.Current.Channel.LocalAddress.Uri;
 var uri =
     isExternalUrl
         ? new Uri(relativeImageUrl)
           : requestedUri;
   var dns =
         String.Concat(
                requestedUri.Scheme,
              "://",
                requestedUri.Host
         );

  if (requestedUri.Port > 0
      && requestedUri.Port > 80)
 {
     dns = String.Concat(dns, ":", requestedUri.Port);
 }

   if (!isExternalUrl) // an image residing within the asset service directory structure
 {
     if (!relativeImageUrl.StartsWith("~/") && !relativeImageUrl.StartsWith("/"))
      {
         relativeImageUrl = String.Concat("~/", relativeImageUrl);
     }
     else if (!relativeImageUrl.StartsWith("~/"))
      {
         relativeImageUrl = String.Concat("~", relativeImageUrl);
      }
 }

   
  // not in cache, attempt to retrieve from the ImageGen XML cache file
 var filename =
        isExternalUrl 
            ? relativeImageUrl
            : Path.GetFileName(relativeImageUrl);
 var xmlCacheFile =
        isExternalUrl
         ? HostingEnvironment.MapPath(String.Format("~/data/Cached/{0}/index.xml", uri.Host))
          : String.Concat(
              Path.GetDirectoryName(HostingEnvironment.MapPath(relativeImageUrl)),
              "\\Cached\\index.xml"
             );

  _logger.Debug("Looking for XML file at '{0}'", xmlCacheFile);
  
 if (!File.Exists(xmlCacheFile)) return null;

    _logger.Debug("Found the XML file");

    var xmlSerializer = new XmlSerializer(typeof(ImageGenCacheIndexEntity));

    using (var reader = new StreamReader(xmlCacheFile))
   {
     // deserialize the XML into an object
     var imageGenCachedIndex = (ImageGenCacheIndexEntity)xmlSerializer.Deserialize(reader);

      if (imageGenCachedIndex != null)
      {
         // find an image which matches the criteria
           var originalImage =
               isExternalUrl
                 ? imageGenCachedIndex.Images
                      .FirstOrDefault(i => i.ImageUrl == filename)
                   : imageGenCachedIndex.Images
                      .FirstOrDefault(i => i.OriginalFileName == filename);

            if (originalImage != null)
            {
             if (imageGenSettings.Constrain && (imageGenSettings.Height > 0 || imageGenSettings.Width > 0))
              {
                 if (originalImage.Width <= originalImage.Height)
                   {
                     ratio = Decimal.Divide(imageGenSettings.Height, originalImage.Height);
                        resizedHeight = imageGenSettings.Height;
                      resizedWidth = Convert.ToInt32(Math.Round(originalImage.Width * ratio));
                  }
                 else
                  {
                     ratio = Decimal.Divide(imageGenSettings.Width, originalImage.Width);
                      resizedHeight = Convert.ToInt32(Math.Round(originalImage.Height * ratio));
                        resizedWidth = imageGenSettings.Width;
                    }
             }
             else
              {
                 resizedHeight = 
                      imageGenSettings.Height > 0
                            ? imageGenSettings.Height
                         : originalImage.Height;
                   resizedWidth = 
                       imageGenSettings.Width > 0
                         ? imageGenSettings.Width
                          : originalImage.Width;
                }

               // todo: add checks for text / background / etc

             var cachedImage =
                 new List<ImageGenCacheIndexCachedImageEntity>(originalImage.CachedImages)
                                    {
                                         new ImageGenCacheIndexCachedImageEntity
                                           {
                                                 FileName = originalImage.OriginalFileName,
                                                Height = originalImage.Height,
                                                Width = originalImage.Width
                                           }
                                     }
                        .FirstOrDefault(
                          i => i.Height == resizedHeight && i.Width == resizedWidth
                      );

              _logger.Debug(String.Format("Original - {0}x{1}", originalImage.Width, originalImage.Height));
                _logger.Debug(String.Format("Requested - {0}x{1}", imageGenSettings.Width, imageGenSettings.Height));
             _logger.Debug(String.Format("Searched - {0}x{1}", resizedWidth, resizedHeight));

                if (cachedImage != null)
              {
                 _logger.Debug("FOUND");

                 if (relativeImageUrl.StartsWith("~")) relativeImageUrl = relativeImageUrl.Substring(1);

                 return
                        isExternalUrl
                         ? String.IsNullOrEmpty(cachedImage.FileName)
                                  ? originalImage.ImageUrl
                                  : String.Concat(dns, "/data/Cached/", uri.Host, "/",cachedImage.FileName)
                         : String.Concat(
                              dns,
                              Path.GetDirectoryName(relativeImageUrl).Replace("\\", "/"),
                               cachedImage.FileName == originalImage.OriginalFileName
                                    ? "/"
                                 : "/Cached/",
                             cachedImage.FileName
                          );
                }

               _logger.Debug("NOT");
         }
     }
 }

   return null;
}

Thank you John for the code. But then there is an issue with ImageGen, I have checked the index.xml about the cahche images being accessed, the information looks correct most of the times, but when I check the file system monitor the file never gets accessed or read from the hard disk. I think the caching is not working properly as the cached images are not being retireved properly. I hope the developer looks into this.

We ended up setting up a reverse proxy of our own (Squid) to cache the ImageGen output

is working on a reply...