Drop Down List for Enum or List in MVC C#

If you do a lot of MVC programming, you are representing objects in the database which many times have one to many relationships.  A great example of this is a type.  You may have a vendor with several types, like drop ship or mail order.  So in the UI, its nice to have a dropdownlist for the type vs. trying to key it correctly every time.  Here is my solution for this.  
First I create a function to load a list of objects I want the drop down list to represent.  Here is an example of a function that returns a list of vendors.

public static IList<IVendor> LoadVendors()
        {
            return Vendor.LoadList();
        }

After that, you add this to your view.

@Html.DropDownListFor(model=>model.VendorId, new SelectList(MyStore.DataBridge.LoadVendors(),"VendorId","Name").OrderBy(c=>c.Text))

You see, The Html.DropDownListFor will associate the value selected from the drop down list to the model.VendorId in this case so on post back reflection to the controller, this value will be reflected in the posted object, so that works great.   The other half is the list, what I found is you can reference any SelectList which I just new up and pass an IEnumerable (in this case the IList<Vendor> returned from my function> with the filed of the object you want to key on, and the field of the object you want to display in the ddl.  Perfect and simple.

To actually do this from a Enum, use the Enum.GetValues function.  It works slick.

@Html.DropDownListFor(model => model.FolderTypeId, new SelectList(Enum.GetValues(typeof(MyEnums.EFolderType))))

HDFS Browser

So I have been working with Hadoop quite a bit lately, and frankly am not a huge fan of the command line interface to the file system nor the web browser functions.  I would love it if there was just a windows explorer type interface to the HDFS.  So what I did, using the webhdfs class in my last post, I created a simple C# windows app that sort of has that windows explorer look and feel to it.  Download the binaries here.

The first thing you will do when running this app (.net 4.5 required) is do file > Connect to HDFS.  This will prompt you for a Server connection which defaults to localhost.  This would be the access node on the cluster.  Then you are prompted for port and username.  Port is defaulted to 50070 and user is blank by default which is the default hdfs user.

Right click in the tree view on a folder gives you the option to delete the folder, create a sub folder, or upload a file.  These are all self explanatory features much like windows explorer would provide.

On the left is a tree view of the directory structure that you can expand and collapse.  Clicking on a directory loads the files in that directory on the right.  You can click on the files on the right to see info in the bottom pane.  If you double click a file, it will attempt to download the file.

Its worth noting that this isn't super fast on my cluster, but I like the interface over the ones provided by hadoop.

Binaries here.
HDFSBrowser.zip (1.5MB)

WebHDFS C# Wrapper with Username Support

So I wanted to do some Hadoop work from C#.  It seems one of the best ways to accomplish this is through the WebHDFS portal.  I started by visiting the WebHDFS page here and created a quick class that wrapped a few functions.  I had to do a lot of foot work to make it decrypt the returned objects, but it worked.  After that I looked around, someone has to have done this, well Microsoft did with their Microsoft.Hadoop.WebHDFS assembly which can be found on their websites.  You know what, it worked well.  I did some coding against it and the only thing it fell short of was exposing the username interface.  Basically, it only used the default username, it did not allow you to connect with a different user.  
What I did was to get the WebHDFSClient class and implemented my own version of it that did.  That class is below.  Now, this class is not stand alone, it returns objects Microsoft defined for the Newtonsoft.Json assembly to reflect the json results from Hadoop into, however, if you already have the other assemblies there, this class called MyWebHDFSClient, will allow you to connect with a username.  In theory, I could create a stand alone class, but I see little benefit to me doing this.

using Newtonsoft.Json.Linq;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Hadoop.WebHDFS;

namespace HDFSBrowser
{

    // based off of : http://hadoop.apache.org/docs/r1.0.0/webhdfs.html

    public class MyWebHDFSClient// : Microsoft.Hadoop.WebHDFS.WebHDFSClient
    {
        //  "http://localhost:50070/webhdfs/v1";
        private string clusterHost = "localhost";
        private string clusterPort = "50070";

        public MyWebHDFSClient() { }

        public MyWebHDFSClient(string clusterHost)
        {
            this.clusterHost = clusterHost;
        }

        public MyWebHDFSClient(string clusterHost, string clusterPort)
        {
            this.clusterHost = clusterHost;
            this.clusterPort = clusterPort;
        }

        public string UserName { get; set; }

        public MyWebHDFSClient(string clusterHost, string clusterPort, string userName)
        {
            this.clusterHost = clusterHost;
            this.clusterPort = clusterPort;
            this.UserName = userName;
        }

        #region "read"

        /// <summary>
        /// List the statuses of the files/directories in the given path if the path is a directory. 
        /// </summary>
        /// <param name="path"></param>
        /// <returns></returns>
        public async Task<DirectoryListing> GetDirectoryStatus(string path)
        {
            JObject files = await GetWebHDFS(path, "LISTSTATUS");
            return new DirectoryListing(files);
        }

        /// <summary>
        /// Return a file status object that represents the path.
        /// </summary>
        /// <param name="path"></param>
        /// <returns></returns>
        public async Task<DirectoryEntry> GetFileStatus(string path)
        {
            JObject file = await GetWebHDFS(path, "GETFILESTATUS");
            return new DirectoryEntry(file.Value<JObject>("FileStatus"));
        }

        /// <summary>
        /// Return the current user's home directory in this filesystem. 
        /// The default implementation returns "/user/$USER/". 
        /// </summary>
        /// <returns></returns>
        public async Task<string> GetHomeDirectory()
        {
            JObject path = await GetWebHDFS("/", "GETHOMEDIRECTORY");
            return path.Value<string>("Path");
        }

        /// <summary>
        /// Return the ContentSummary of a given Path
        /// </summary>
        /// <param name="path"></param>
        /// <returns></returns>
        public async Task<ContentSummary> GetContentSummary(string path)
        {
            JObject contentSummary = await GetWebHDFS(path, "GETCONTENTSUMMARY");
            return new ContentSummary(contentSummary.Value<JObject>("ContentSummary"));
        }

        /// <summary>
        /// Get the checksum of a file
        /// </summary>
        /// <param name="path">The file checksum. The default return value is null, which 
        /// indicates that no checksum algorithm is implemented in the corresponding FileSystem. </param>
        /// <returns></returns>
        public async Task<FileChecksum> GetFileChecksum(string path)
        {
            JObject fileChecksum = await GetWebHDFS(path, "GETFILECHECKSUM");
            return new FileChecksum(fileChecksum.Value<JObject>("FileChecksum"));
        }

        // todo, overloads with offset & length
        /// <summary>
        /// Opens an FSDataInputStream at the indicated Path
        /// </summary>
        /// <param name="path"></param>
        /// <returns></returns>
        public async Task<HttpResponseMessage> OpenFile(string path)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.GetAsync(GetRootUri() + path + "?op=OPEN" + GetCredentials());
            resp.EnsureSuccessStatusCode();
            return resp;
        }

        /// <summary>
        /// Opens an FSDataInputStream at the indicated Path.  The offset and length will allow 
        /// you to get a subset of the file.  
        /// </summary>
        /// <param name="path"></param>
        /// <param name="offset">This includes any header bytes</param>
        /// <param name="length"></param>
        /// <returns></returns>
        public async Task<HttpResponseMessage> OpenFile(string path, int offset, int length)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.GetAsync(GetRootUri() + path + "?op=OPEN&offset=" + offset.ToString() + "&length=" + length.ToString() + GetCredentials());
            resp.EnsureSuccessStatusCode();
            return resp;
        }

        #endregion

        #region "put"
        // todo: add permissions
        /// <summary>
        /// Make the given file and all non-existent parents into directories. 
        /// Has the semantics of Unix 'mkdir -p'. Existence of the directory hierarchy is not an error. 
        /// </summary>
        /// <param name="path"></param>
        /// <returns></returns>
        public async Task<bool> CreateDirectory(string path)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=MKDIRS" + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            var content = await resp.Content.ReadAsAsync<JObject>();
            return content.Value<bool>("boolean");

        }

        /// <summary>
        /// Renames Path src to Path dst.
        /// </summary>
        /// <param name="path"></param>
        /// <param name="newPath"></param>
        /// <returns></returns>
        public async Task<bool> RenameDirectory(string path, string newPath)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=RENAME&destination=" + newPath + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            var content = await resp.Content.ReadAsAsync<JObject>();
            return content.Value<bool>("boolean");

        }

        /// <summary>
        /// Delete a file.  Note, this will not recursively delete and will
        /// not delete if directory is not empty
        /// </summary>
        /// <param name="path">the path to delete</param>
        /// <returns>true if delete is successful else false. </returns>
        public Task<bool> DeleteDirectory(string path)
        {
            return DeleteDirectory(path, false);
        }


        /// <summary>
        /// Delete a file
        /// </summary>
        /// <param name="path">the path to delete</param>
        /// <param name="recursive">if path is a directory and set to true, the directory is deleted else throws an exception.
        /// In case of a file the recursive can be set to either true or false. </param>
        /// <returns>true if delete is successful else false. </returns>
        public async Task<bool> DeleteDirectory(string path, bool recursive)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.DeleteAsync(GetRootUri() + path + "?op=DELETE&recursive=" + recursive.ToString().ToLower() + GetCredentials());
            resp.EnsureSuccessStatusCode();
            var content = await resp.Content.ReadAsAsync<JObject>();
            return content.Value<bool>("boolean");
        }

        /// <summary>
        /// Set permission of a path.
        /// </summary>
        /// <param name="path"></param>
        /// <param name="permissions"></param>
        public async Task<bool> SetPermissions(string path, string permissions)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=SETPERMISSION&permission=" + permissions + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            return true;
        }

        /// <summary>
        /// Sets the owner for the file 
        /// </summary>
        /// <param name="path"></param>
        /// <param name="owner">If it is null, the original username remains unchanged</param>
        public async Task<bool> SetOwner(string path, string owner)
        {
            // todo, add group
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=SETOWNER&owner=" + owner + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            return true;
        }

        /// <summary>
        /// Sets the group for the file 
        /// </summary>
        /// <param name="path"></param>
        /// <param name="group">If it is null, the original groupname remains unchanged</param>
        public async Task<bool> SetGroup(string path, string group)
        {
            // todo, add group
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=SETOWNER&group=" + group + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            return true;
        }

        /// <summary>
        /// Set replication for an existing file.
        /// </summary>
        /// <param name="path"></param>
        /// <param name="replicationFactor"></param>
        /// <returns></returns>
        public async Task<bool> SetReplicationFactor(string path, int replicationFactor)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=SETREPLICATION&replication=" + replicationFactor.ToString() + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            var content = await resp.Content.ReadAsAsync<JObject>();
            return content.Value<bool>("boolean");
        }

        /// <summary>
        /// Set access time of a file
        /// </summary>
        /// <param name="path"></param>
        /// <param name="accessTime">Set the access time of this file. The number of milliseconds since Jan 1, 1970. 
        /// A value of -1 means that this call should not set access time</param>
        public async Task<bool> SetAccessTime(string path, string accessTime)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=SETTIMES&accesstime=" + accessTime + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            return true;
        }

        /// <summary>
        /// Set modification time of a file
        /// </summary>
        /// <param name="path"></param>
        /// <param name="modificationTime">Set the modification time of this file. The number of milliseconds since Jan 1, 1970.
        /// A value of -1 means that this call should not set modification time</param>
        public async Task<bool> SetModificationTime(string path, string modificationTime)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.PutAsync(GetRootUri() + path + "?op=SETTIMES&modificationtime=" + modificationTime + GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            return true;
        }

        /// <summary>
        /// Opens an FSDataOutputStream at the indicated Path. Files are overwritten by default.
        /// </summary>
        /// <param name="localFile"></param>
        /// <param name="remotePath"></param>
        /// <returns></returns>
        public async Task<string> CreateFile(string localFile, string remotePath)
        {
            WebRequestHandler wrc = new WebRequestHandler { AllowAutoRedirect = false };
            HttpClient hc = new HttpClient(wrc);
            var resp = await hc.PutAsync(GetRootUri() + remotePath + "?op=CREATE" + GetCredentials(), null);
            var putLocation = resp.Headers.Location;
            StreamContent sc = new StreamContent(System.IO.File.OpenRead(localFile));
            var resp2 = await hc.PutAsync(putLocation, sc);
            resp2.EnsureSuccessStatusCode();
            return resp2.Headers.Location.ToString();
        }

        /// <summary>
        /// Append to an existing file (optional operation).
        /// </summary>
        /// <param name="localFile"></param>
        /// <param name="remotePath"></param>
        /// <returns></returns>
        public async Task<string> AppendFile(string localFile, string remotePath)
        {
            WebRequestHandler wrc = new WebRequestHandler { AllowAutoRedirect = false };
            HttpClient hc = new HttpClient(wrc);
            var resp = await hc.PostAsync(GetRootUri() + remotePath + "?op=APPEND" +  GetCredentials(), null);
            resp.EnsureSuccessStatusCode();
            var postLocation = resp.Headers.Location;
            StreamContent sc = new StreamContent(System.IO.File.OpenRead(localFile));
            var resp2 = await hc.PostAsync(postLocation, sc);
            resp2.EnsureSuccessStatusCode();
            // oddly, this is returning a 403 forbidden 
            // due to: "IOException","javaClassName":"java.io.IOException","message":"java.io.IOException: 
            // Append to hdfs not supported. Please refer to dfs.support.append configuration parameter.
            return resp2.Headers.Location.ToString();
        }

        #endregion

        public string GetCredentials()
        {
            if (string.IsNullOrEmpty(UserName))
                return string.Empty;
            else
                return "&user.name=" + UserName;
        }

        public string GetRootUri()
        {
            // todo: move to some config based mechanism
            //  "http://localhost:50070/webhdfs/v1";
            return "http://" + clusterHost + ":" + clusterPort + "/webhdfs/v1";
        }

        private async Task<JObject> GetWebHDFS(string path, string operation)
        {
            HttpClient hc = new HttpClient();
            var resp = await hc.GetAsync(GetRootUri() + path + "?op=" + operation + GetCredentials());
            resp.EnsureSuccessStatusCode();
            JObject files = await resp.Content.ReadAsAsync<JObject>();
            return files;
        }
    }
}

Setting up a WCF Endpoint with SSL

So you want to have your web service communicate with SSL, well here is how you can set it up in the config file easily.  First you need to go into your config file and make sure your service is added.  In it, you should have an endpoint, I use the default name soapService below.  Next you need to reference your binding which I use a basicHttpBinding.  As you can see below that, I have a binding declared in the basicHttpBindings called secureHttpBinding with a security mode of Transport and the clientCredentialType of None as I am not concerned with authentication at this point.  This is named secureHttpBinding which I reference back in the endpoint under the attribute bindingConfiguration.  Finally, I have a behavior referenced by the service behaviorConfiguration called ServBehave.  The behavior is configured to have httpsGetEnabled set to true and I have service debug turned on in this code, a production system would have that off.  

<system.serviceModel>
    <services>
      <service name="PPMRK_PLMService.PLMService" behaviorConfiguration="ServBehave">
        <!--CustomValidator"-->
        <endpoint
					address="soapService"
					binding="basicHttpBinding"
          bindingConfiguration="secureHttpBinding"
					contract="PPMRK_PLMService.IPLMService"
					/>
      </service>
    </services>
    <bindings>
      <basicHttpBinding>
        <binding name="secureHttpBinding">
          <security mode="Transport">
            <transport clientCredentialType="None"/>
          </security>
        </binding>
      </basicHttpBinding>      
    </bindings>
    <behaviors>
      <serviceBehaviors>
        <behavior name="ServBehave">
          <serviceMetadata httpsGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <protocolMapping>
        <add binding="basicHttpsBinding" scheme="https" />
    </protocolMapping>    
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="false" />
  </system.serviceModel>
There you have it, a simple way to have https enabled on your web service.  If you wanted http also enabled or through a different endpoint, you would have to set that up in parallel with a different name.

Doubles and Scientific Notation in C#

I created a math engine for evaluating equations a while ago and ran into some issues with scientific notation.  My parsers got hung up on it in short because default ToString() on a double is the shortest result.  So 0.000001 is longer that 1E-6 so the latter is used, which would be ok except casting 1E-6 to a decimal results in an exception.  "Input string was not in a correct format."  So that sucks. 

After some googling, I found that if you use the format string N in to string, you can follow it with the number of decimals to round to, and it won't do scientific.  And to be honest, you can only be so precise with most numbers so you can predict the precision needed or usable in most instances.  So var.ToString("N7") will return 0.000001 instead of scientific notation.