Friday, September 7, 2012

Playing with Google Search Results

You will need:

Create a Visual Studio project, for example C# Windows Forms application. Drop a TextBox, a Button and a ListView on the form. Creat a class for the methods to be used, let's say Helper.cs. First, I'm using the System.Net.Webclient to call Google and get a page of search results.

public static WebClient webClient = new WebClient();

public static string GetSearchResultHtlm(string keywords)
{
    StringBuilder sb = new StringBuilder("http://www.google.com/search?q=");
    sb.Append(keywords);
    return webClient.DownloadString(sb.ToString());
}

The string that is returned is the html of the first page of the Google search for the string that is passed to the method. Opened in the web browser, it will look something like this

Google search result page

What I want to extract is the actual links, which are marked in red on the screenshot above. Here I'm going to use HtmlAgilityPack to load the string into the HtmlDocument object. After the string is loaded, I will use a simple LINQ query to extract the nodes that match certain conditions: They are html links (a href), and the URL of the link contains either "/url?" or "?url=". By this point, I get quite and unreadable list of values.

Raw URLs

To bring it into readable form, I'll match it to a regular expression and then load the results into the ListView. Here is the code:

public static Regex extractUrl = new Regex(@"[&?](?:q|url)=([^&]+)", RegexOptions.Compiled);

public static List<String> ParseSearchResultHtml(string html)
{
    List<String> searchResults = new List<string>();

    var doc = new HtmlAgilityPack.HtmlDocument();
    doc.LoadHtml(html);

    var nodes = (from node in doc.DocumentNode.SelectNodes("//a")
                 let href = node.Attributes["href"]
                 where null != href
                 where href.Value.Contains("/url?") || href.Value.Contains("?url=")
                 select href.Value).ToList();

    foreach (var node in nodes)
    {
        var match = extractUrl.Match(node);
        string test = HttpUtility.UrlDecode(match.Groups[1].Value);
        searchResults.Add(test);
    }

    return searchResults;
}

Here is the result:

Final Results

I'm not quite sure why this may be useful, but as an exercise it is possible to add an option to parse through a certain number of pages, rather than just the first page. But if you try to run those queries in an automated mode, Google will soon start serving 503 errors to you.

by . Also posted on my website

Sunday, August 12, 2012

Securing Access to Windows 7 Folder from Everyone but a Single User

Today I had to perform a fairly specific task: restrict access to a Windows 7 folder to a single user. I think I found the way to do it properly, and it is not a straightforward task. Before I forget, I might keep a record of all actions required because I did not find a clear sequence anywhere on the net. It will only take 10 easy steps.

Let's assume there is a folder called Bob's Documents where only Bob, and not even the Admin>, should have access.

  1. Right-click on Bob's Documents and select Properties
  2. Select Bob's Documents Properties

  3. Bob's Documents Properties window will open. Switch to Security tab and click Advanced button.
  4. Bob's Documents Properties

  5. Advanced Security Settings for Bob's Documents will open. On the Permissions tab, Click Change Permissions button.
  6. Advanced Security Settings for Bob's Documents

  7. Another window will open. Unfortunately, it's too called Advanced Security Settings for Bob's Documents, adding to confusion. In this new window, untick Include inheritable permissions from this object's parent - that will simplify things a lot, because we only care about permissions to this folder, not its parent folder.
  8. Advanced Security Settings for Bob's Documents - but not the same one!

  9. As soon as the chechbox is unticked, a warning called Windows Security will pop up. Since we're getting rid of parent permissions, click Remove.
  10. Windows Security warning

  11. All permissions should have disappeared from the Permission entries. Now click Add.
  12. Select User or Group window will open. In Enter the object name to select, type Bob and click Check Names to make sure there is no typo. Bob's name should resolve to PCName\Bob.
  13. Select User or Group

  14. Click OK. Now the Permissions Entry for Bob's Documents window will pop up. Let's give Bob full control - click the checkbox across from Full Control under Allow. All other checkboxes under Allow will select automatically. Click OK to close this window.
  15. Permissions Entry for Bob's Documents

  16. About done. Click OK in Advanced Security Settings for Bob's Documents to close it, and in another Advanced Security Settings for Bob's Documents to close it too, and OK in Bob's Documents Properties.
  17. Try to browse to Bob's Documents. Even if you're on Administrator account, you should not be able to, but you should if you are logged in as Bob.
  18. Permissions are set

by . Also posted on my website

Thursday, August 9, 2012

Learning MVC: Display a Custom Error Page, Log Error and Send Email

Step one - create my own controller class. Simple, just add a BaseController to the Controllers folder

public abstract class BaseController : Controller
{
}

and then modify all existing contollers to inherit from BaseController rather than from System.Web.Mvc.Controller.

Next, I override the OnException method in the BaseController which is called whenever the exception is thrown within an action.

public abstract class BaseController : Controller
{
    protected override void OnException(ExceptionContext filterContext)
    {
        var fileName = Path.Combine(Request.MapPath("~/App_Data"), "log.txt");
        WriteLog(fileName, filterContext.Exception.ToString());
        filterContext.ExceptionHandled = true; 
        this.View("Error").ExecuteResult(this.ControllerContext);
    }
    
    static void WriteLog(string logFile, string text)
    {   
        StringBuilder message = new StringBuilder();
        message.AppendLine(DateTime.Now.ToString());
        message.AppendLine(text);
        message.AppendLine("=========================================");
        System.IO.File.AppendAllText(logFile, message.ToString());
    }
}

I can verify and find out that the Yellow Screen of Death is not indeed shown

Custom Error Page

And the log file is in my App_Data folder

Log File

Now that I can see it works, I would still want to see the exceptions as soon as they occur, rather than checking the log file on each occasion. To achieve that, first I need to add

<customErrors mode="RemoteOnly" />

to the system.web section of the Web.config file and then in the OnException method check if this section is set.

protected override void OnException(ExceptionContext filterContext)
{
    var fileName = Path.Combine(Request.MapPath("~/App_Data"), "log.txt");
    WriteLog(fileName, filterContext.Exception.ToString());
    if (filterContext.HttpContext.IsCustomErrorEnabled)
    {
        filterContext.ExceptionHandled = true; 
        this.View("Error").ExecuteResult(this.ControllerContext);
    }
}

Finally, I would like to receive an email when something on my website goes wrong. I'm adding a function for that, rearranging a few lines and come up with the final version of my BaseController (for now).

using System;
using System.Web.Mvc;
using System.Text;
using System.IO;
using System.Net.Mail;

namespace Recipes.Controllers
{
    public abstract class BaseController : Controller
    {
        protected override void OnException(ExceptionContext filterContext)
        {
            string ex = filterContext.Exception.ToString();
            var fileName = Path.Combine(Request.MapPath("~/App_Data"), "log.txt");
            WriteLog(fileName, ex);
            SendEmail(ex);
            if (filterContext.HttpContext.IsCustomErrorEnabled)
            {
                filterContext.ExceptionHandled = true; 
                this.View("Error").ExecuteResult(this.ControllerContext);
            }
        }

        static StringBuilder ErrorText(string text)
        {
            StringBuilder message = new StringBuilder();
            message.AppendLine(DateTime.Now.ToString());
            message.AppendLine(text);
            message.AppendLine("=========================================");
            return message;
        }

        static void WriteLog(string logFile, string text)
        {
            System.IO.File.AppendAllText(logFile, ErrorText(text).ToString());
        }

        static void SendEmail(string text)
        {
            MailMessage mail = new MailMessage();
            SmtpClient client = new SmtpClient("smtp.example.com");
            client.Credentials = new System.Net.NetworkCredential("u$3r", "pa$$word"); client.Port = 587;

            mail.From = new MailAddress("mvc@example.com");
            mail.To.Add("developer@example.com");
            mail.Subject = "Error on your website";
            mail.Body = ErrorText(text).ToString();

            client.Send(mail); 
        }
    }
}

References

Problem with generic base controller error handling in ASP.NET MVC
ASP.NET MVC HandleError Attribute, Custom Error Pages and Logging Exceptions
How to send email from Asp.net Mvc-3?
by . Also posted on my website

Monday, August 6, 2012

Learning MVC: Game of Life in MVC

I wrote some code and made a quick WPF application that implemented Conway's Game of Life earlier ( Game of Life Exercise and Extension Methods).

Next, I wanted to see how the game could be run on the MVC platform. The solution in short: use javaScript setInterval function to load the partial view into the div. Use a method in the controller to generate the partial view.

Here is how my solution looked:

public class GameController : Controller
{
    public ActionResult Index()
    {
        GameOfLifeHelpers.InitializeGame();
        return View(NextIteration());
    }

    [OutputCache(NoStore = true, Location = OutputCacheLocation.Client, Duration = 1)]
    public ActionResult Update()
    {
        return PartialView("_Table", NextIteration());
    }

    public HtmlString NextIteration()
    {
        GameOfLifeHelpers.DrawNextIteration();
        return new HtmlString(GameOfLifeHelpers.table.ToString());
    }
}

The partial view is called _Table and is nothing more than the HtmlString. Here is the partial view:

@model HtmlString
           
@{ Layout = null; }

@Model

The model is just the HtmlString which gets rendered, and the HtmlString itself is just a simple table of a specified number of cells. And here is the Index.cshtml

<script type="text/javascript">
    setInterval("$('#update').load('/Game/Update')", 1000);
</script>

@model HtmlString

@{
    ViewBag.Title = "Index";
}

<h2>Index</h2>
<div id="update">@{Html.RenderPartial("_Table", Model);}</div>

Note how the interval is set to 1000 ms and the OutputCache duration in the controller is set to the same value. Every second the call to load will return a partial view from the Update method. What does the Update method return? When the game starts, and empty html table is created with each cell having a blue background.

public static void NewGameTable()
{
    table = new StringBuilder(@"<table border=1 bordercolor=black cellspacing=0 cellpadding=0>");
    for (int i = 0; i < y; i++)
    {
        table.Append("<tr>");
        for (int j = 0; j < x; j++)
        {
            table.Append("<td width=10px height=10px bgcolor=#0276FD></td>");
        }
        table.Append("</tr>");
    }
    table.Append("</table>");
}

Then, on each iteration, a new boolead array is filled to specify which cells will be "alive".

public static void DrawNextIteration()
{
    bool[,] arrCurrent = counter % 2 == 0 ? arrOne : arrTwo;
    bool[,] arrNext = counter % 2 == 0 ? arrTwo : arrOne;
    FillArray(arrNext, arrCurrent);
    counter++;
    for (int i = 0; i < y; i++)
    {
        for (int j = 0; j < x; j++)
        {
            if (arrNext[i, j] != arrCurrent[i, j])
            {
                table = arrNext[i, j] ? GameOfLifeTableReplaceCell(i, j, "#FF0000", table) : GameOfLifeTableReplaceCell(i, j, "#0276FD", table);
            }
        }
    }
}

The function that replaces the cell is very simple - it calculates the position where the font for the cell is specified based on the coordinates and makes the cell color red if it went from "dead" to "alive", and vice versa.

public static StringBuilder GameOfLifeTableReplaceCell(int i, int j, string colour, StringBuilder sb)
{
    const int rowLength = 48*x + 9;
    const int cellLength = 48;
    int start = 62 + i * rowLength + 4 + j * cellLength + 35;
    sb.Remove(start, 7);
    sb.Insert(start, colour);
    return sb;
}

The rest of the code is omitted because it can be found in my earlier post and used with little or no change.

by . Also posted on my website

Sunday, August 5, 2012

The Case of a Strangely Coloured ProgressBar

At first this bug report puzzled me a bit. Essentially it said "Progress bar fills with blue rectangles, status text nearly impossible to read". That was a case of "works on my machine" because all I could see was that:

Can you see blue rectangles?

However, I soon discovered that just optimizing my Windows visual effects for best performance does a neat trick. If you don't know, it's under Control Panel → System → Advanced System Settings → Advanced → Performance → Settings → Visual Effects → Adjust for best performance, as in the screen shot below

Visual Effects

Here is what I saw:

Can you see blue rectangles NOW?

That was an easy fix. I just modified the ForeColor of the ProgressBar as shown below from the default value:

//before
<ProgressBar Name="Progress" Grid.Column="0" Value="{Binding ProgressValue}" HorizontalAlignment="Stretch"/>

//after
<ProgressBar Foreground="LightGreen" Name="Progress" Grid.Column="0" Value="{Binding ProgressValue}" HorizontalAlignment="Stretch"/>

Here is what I saw after making this change:

Adjusted for best performance

Allow Windows to determine settings

Reference:

WPF: Progressbar foreground colorby . Also posted on my website

Saturday, August 4, 2012

Installing Windows Updates via Shell Script

I had to dive into the world of shell scripting and do some work there. One of the tasks I had was to automatically install a bunch of Windows Updates on a server. The server could not be connected to the Internet so the updates were provided as separate files. The updates could be a mix of regular executables or Microsoft Update Standalone Packages (.msu). The script gets the name of the folder it runs from and then iterates over the files in this folder. It checks the file extension and runs appropriate command depending on the file being EXE or MSU. It also checks the return value and keeps a counter on the number of updates that reported successful and unsuccessful result, and writes the result of each install into a log file. At the end it displays a message that informs a user about the number of successfully installed and failed updates.

Sub Main

Dim objfso, objShell
Dim iSuccess, iFail
Dim folder, files, sFolder, folderidx, Iretval, return
Set objfso = CreateObject("Scripting.FileSystemObject")
Set objShell = CreateObject("Wscript.Shell")
 
sFolder = left(WScript.ScriptFullName,(Len(WScript.ScriptFullName))-(len(WScript.ScriptName)))
Set folder = objfso.GetFolder(sFolder)
Set logFile = objfso.CreateTextFile("C:\log.txt", TRUE)
Set files = folder.Files
iSuccess = 0
iFail = 0
 
For each folderIdx In files
 If Ucase(Right(folderIdx.name,3)) = "MSU" then
  logFile.WriteLine("Installing " & folderidx.name & "...")
  wscript.echo "Installing " & folderidx.name & "..."
  iretval=objShell.Run ("wusa.exe " & sfolder & folderidx.name & " /quiet /norestart", 1, True)
  If (iRetVal = 0) or (iRetVal = 3010) then
   logFile.WriteLine("Success.")
   wscript.echo "Success."
   iSuccess = iSuccess + 1
  Else
   logFile.WriteLine("Failed.")
   wscript.echo "Failed."
   iFail = iFail + 1
  End If
 ElseIf Ucase(Right(folderIdx.name,3)) = "EXE" Then
  logFile.WriteLine("Installing " & folderidx.name & "...")
  wscript.echo "Installing " & folderidx.name & "..."
  iretval = objShell.Run(folderidx.name & " /q /norestart", 1, True)
  If (iRetVal = 0) or (iRetVal = 3010) then
   logFile.WriteLine("Success.")
   wscript.echo "Success."
   iSuccess = iSuccess + 1
  Else
   logFile.WriteLine("Failed.")
   wscript.echo "Failed."
   iFail = iFail + 1
  End If
 End If
Next

wscript.echo iSuccess & " update(s) installed successfully and " & iFail & " update(s) failed. See C:\log.txt for details."
 
End Sub

Main()
by . Also posted on my website

Tuesday, July 31, 2012

Game of Life Exercise and Extension Methods

A quick attempt in writing the Game of Life simulator using WPF. As a side goal, I wanted to better understand the extension methods in C# so I tried to move as much code as possible into the Helper class without sacrificing readability.

PopulateGrid adds 50 rows and columns to a WPF Grid, creating a 50x50 matrix of cell. It then adds a rectangle to each cell so coloring could be applied. RePaintCell changes the background color of the cell at position i,j. InitializeArray just fills a 50x50 array of booleans, each value representing either a live or dead cell. CheckCell checks a single cell to find out if it will be live or dead in the next iteration. FillArray uses CheckCell to analyse the current array and construct the array of the next iteration. DrawArray compares the current and next iterations. If there is difference in the color of the cell at i,j, this cell is painted appropriately, otherwise it is skipped. Finally, AddGlider adds a glider element to the empty array so it was easy to test that the game runs correctly. The full listing is below.

MainWindow.xaml

<Window x:Class="GameOfLife.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="386" Width="525">
    <Grid Name="MainGrid">
        <Grid Name="DynamicGrid" ShowGridLines="True">
        </Grid>
        <Button Content="Button" Height="23" HorizontalAlignment="Left" Margin="25,316,0,0" Name="button1" VerticalAlignment="Top" Width="75" Click="button1_Click" />
    </Grid>
</Window>

MainWindow.xaml.cs

using System.Windows;
using System.Windows.Media;
using System.Windows.Threading;
using System;

namespace GameOfLife
{
    public partial class MainWindow : Window
    {
        private bool[,] arrOne = new bool[Helper.x, Helper.y];
        private bool[,] arrTwo = new bool[Helper.x, Helper.y];
        private DispatcherTimer dt = new DispatcherTimer();
        private int count;

        public MainWindow()
        {
            InitializeComponent();
            dt.Interval = TimeSpan.FromMilliseconds(100);
            dt.Tick += dt_Tick;
            dt.Start();
            Loaded += MainWindow_Loaded;
        }

        void MainWindow_Loaded(object sender, RoutedEventArgs e)
        {
            DynamicGrid.PopulateGrid(new SolidColorBrush(Colors.Blue));
            InitializeGame();
        }

        void InitializeGame()
        {
            arrOne.InitializeArray(false);
            arrTwo.InitializeArray(false);
            arrOne.AddGlider(20, 30);
            arrOne.DrawArray(arrTwo, DynamicGrid);
        }

        void dt_Tick(object sender, EventArgs e)
        {
            if(true)
            {
                bool[,] arrCurrent = count%2 == 0 ? arrOne : arrTwo;
                bool[,] arrNext = count%2 == 0 ? arrTwo : arrOne;

                arrNext.FillArray(arrCurrent);
                arrNext.DrawArray(arrCurrent, DynamicGrid);
                count++;
            }
        }

        private void button1_Click(object sender, RoutedEventArgs e)
        {
            if(dt.IsEnabled)
                dt.Stop();
            else
                dt.Start();
        }
    }
}

Helper.cs

using System.Linq;
using System.Windows.Controls;
using System.Windows.Shapes;
using System.Windows.Media;

namespace GameOfLife
{
    public static class Helper
    {
        public const int x = 50;
        public const int y = 50;

        public static void PopulateGrid(this Grid grid, SolidColorBrush brush)
        {
            for (int i = 0; i < y; i++)
            {
                grid.RowDefinitions.Add(new RowDefinition());
            }

            for (int j = 0; j < x; j++)
            {
                grid.ColumnDefinitions.Add(new ColumnDefinition());
            }

            for (int i = 0; i < y; i++)
            {
                for (int j = 0; j < x; j++)
                {
                    Rectangle rect = new Rectangle();
                    rect.SetValue(Grid.RowProperty, i);
                    rect.SetValue(Grid.ColumnProperty, j);
                    rect.Fill = brush;
                    grid.Children.Add(rect);
                }
            }
        }

        public static void RePaintCell(this Grid grid, int i, int j, SolidColorBrush brush)
        {
            Rectangle cell = grid.Children.Cast<Rectangle>().First(r => Grid.GetRow(r) == j && Grid.GetColumn(r) == i);
            cell.Fill = brush;
        }

        public static void InitializeArray<T>(this T[,] arr, T value)
        {
            int iDim = arr.GetLength(0);
            int jDim = arr.GetLength(1);
            for (int i = 0; i < iDim; i++)
            {
                for (int j = 0; j < jDim; j++)
                {
                    arr[i, j] = value;
                }
            }
        }

        public static void AddGlider(this bool[,] arr, int x, int y)
        {
            arr[x - 1, y] = false;
            arr[x - 1, y + 1] = false;
            arr[x - 1, y + 2] = true;

            arr[x, y] = true;
            arr[x, y + 1] = false;
            arr[x, y + 2] = true;

            arr[x + 1, y] = false;
            arr[x + 1, y + 1] = true;
            arr[x + 1, y + 2] = true;
        }

        public static void FillArray(this bool[,] arr, bool[,]arrCurrent)
        {
            for (int i = 0; i < y; i++)
            {
                for (int j = 0; j < x; j++)
                {
                    arr[i, j] = arrCurrent.CheckCell(i, j);
                }
            }
        }

        public static void DrawArray(this bool[,] arr, bool[,] arrCurrent, Grid grid)
        {
            SolidColorBrush redBrush = new SolidColorBrush(Colors.Red);
            SolidColorBrush blueBrush = new SolidColorBrush(Colors.Blue);

            for (int i = 0; i < y; i++)
            {
                for (int j = 0; j < x; j++)
                {
                    if (arr[i, j] != arrCurrent[i, j])
                    {
                        SolidColorBrush brush = arr[i, j] ? redBrush : blueBrush;
                        grid.RePaintCell(i, j, brush);
                    }
                }
            }
        }

        public static bool CheckCell(this bool[,] arr, int i, int j)
        {
            int nextI = i == (x - 1) ? 0 : i + 1;
            int prevI = i == 0 ? x - 1 : i - 1;
            int nextJ = j == (y - 1) ? 0 : j + 1;
            int prevJ = j == 0 ? y - 1 : j - 1;

            bool[] neighbours = new[]{
                                        arr[prevI, prevJ],   arr[i, prevJ],   arr[nextI, prevJ],
                                        arr[prevI, j],       arr[nextI, j],   arr[prevI, nextJ],
                                        arr[i, nextJ],       arr[nextI, nextJ]
                                    };

            int val = neighbours.Count(c => c);

            if (arr[i, j])
                return (val >= 2 && val <= 3) ? true : false;
            else
                return (val == 3) ? true : false;
        }
    }
}

Reference

Conway's Game of Life

The results are impressive

by . Also posted on my website