Setting up a Hadoop Cluster for Hadoop Install

A project I have recently been a part of was setting up a POC hadoop cluster for our organization.  We did a simple 6 node cluster all using CentOS 6.4 and the free Cloudera CDH4.  I had installed this on a single node easy enough with the online tutorials, but felt a comprehensive list of tasks needed to prepare the cluster for install wasn't clearly laid out anywhere.  This is how we did it.  For now its just a list of instructions, I will come back an add screen shots if I ever do it again, this was mostly done through putty with vi for editing.

1. Disable the selinux firewall on all nodes.
ls etc/selinux --go to the folder
vi config --edit the config file
selinux=disable --to disable the linux firewall and allow ports to open up so nodes can talk to eachother.
sudo service network restart --to restart the adapter

2. Generate SSH Key - as root on name node
ssh-keygen -t rsa--enter a few times, key generated
cd .ssh -- to the ssh files generated
cp id_rsa authorized_keys --copy
ssh-copy-id hdfsnode2--copy the key to the node hdfsnode2 (computer network name)
ssh hdfsnode2should now allow login to that machine without password as the key is installed.
repeat on all machines in node

3. IP Table Services
service iptables status
--should see firewall is not running
chkconfig iptables off --now the nodes will not have this service start
repeat on each node
4. Make sure host files one each has all nodes

vi /etc/hosts --update host file to have reference to each node including itself ip fullname alias
scp /etc/hosts hdfsnode2:/etc/ --copy host file to each node

5. Restart all servers
ssh hdfsnode2 --get to other server
init 6 --does remote restart

Adding CKEditor To an MVC 3 Razor in C# with JQuery

I am creating a web application that will allow content manipulation by end users.  This requires me to insert some sort of editor for them to manage content.  I have used several of these in the past on a few of my ASP.Net sites, but this will the first time in MVC 4 with Razor and its been a few years so I decided to look around.  
I stumbled across CKEdit which is a JQuery based library for a simple HTML editor.  The code seemed laid out well so strapping it on MVC shouldn't be that hard, and it wasn't.  Here are the few steps it took me to get this working well.
1. Download CKEdit from their website.
From here you can even choose the layout you like.  It downloads as a zip file that extracts into a folder you should just place at the root level of your web application.
2. Add the new folder and content to you Project
In VS 2012, you must add each folder then the files in it, sort of annoying you can't add an existing folder, oh well.  Make sure to right click the folder and in properties, set it to always copy to output.  
3. Update your Layout file 
If you want this available on all your web pages do it here, otherwise these just go in the pages you need the editor.  Add or make sure these 3 lines are added in the header.  These include all the Javascript you need to make this happen.
@Scripts.Render("~/bundles/jquery")
@Scripts.Render("~/ckeditor/ckeditor.js")
@Scripts.Render("~/ckeditor/adapters/jquery.js")
4. Add this function JS Function on the page you want it rendered.
Again, this could be in an include file or in the layout page, or just on the page you need it.
<script type="text/javascript">
    $(function () {
        $('#Content').ckeditor();
    });
</script>
5. Tie your model to the Editor.
So, the above JavaScript function will replace an object with the name content with the ckeditor.  The attribute on my model is named Content, so the id of it will be content, and so the above function finds it and places the editor over it.  I use the following code to generate a text area for it to replace, mostly because a text area is easy to fail to if javascript isn't working.
<div class="editor-label">
            @Html.LabelFor(model => model.Content)
        </div>
        <div class="editor-field">
            @Html.TextAreaFor(model => model.Content)
            @Html.ValidationMessageFor(model => model.Content)
        </div>
6. Update Controller
My controller now needs to be able to land my post with the model.  What is interesting about this is it will throw an error because of possible injection allowing all characters to be posted to the server.  To allow the editor to post the HTML, which it must or it will never render later, you need to put the [ValidateInput(false)] above the function that catches it.  After that, treat it like a string and save it away.
7. Render the saved content.
This should be as easy as just @Object.Field however it isn't of course.  MVC is smarter than us, and html encodes any string rendered so all tags will show up.  What we have to do instead is @Html.Raw(Object.Field) and everything will look great.  
8 Have Fun!

Shipping Girth Calculations in C#

I recently had to calculate girth in a shipping sense which isn't very exciting.  It requires you to take the length, width, and height of the object and use the following equation.  Longest Side + 2*(Middle Side + Shortest Side).  
I would like to do this in a line of code, but alas, determining which of the 3 variables is longest makes this a if else type problem.  Here is my solution, I thought it rather nice, though I'm sure more memory is used and it compiled larger than a simple if else setup, but I think it reads nicer.

var sl = new List<double>() { Length, Width, Height };
var Girth = sl.Max() + 2 * sl.OrderBy(c=>c).ToList()[1] + 2 * sl.Min();

The use of a List allows some macro functions like Max and Min, and I can use the .orderby function and some lambda to index through and get the middle sized item.

WCF ws-Security and SoapUI

So recently I talked about ws-Security Headers with WCF.  http://www.ghij.org/blog/post/2014/04/01/username-wssecurity-header-with-wcf.aspx  But today I ran into the problem of trying to test said service locally.  I like to use soapUI which is a great tool.  The problem I ran into was it kept failing saying with the error BadContextToken from WCF.  Some time on google showed this is a common problem with SoapUI connecting to WCF with the UserName security type. 
In order to still get my testing done, I ended up creating a second endpoint with its own binding to do testing with SoapUI. Obviously you wouldn't release this for the security hole it creates, but locally and on dev, it allows you nice debugging of the service with SoapUI.
1. Create the new endpoint (next to the old one)
<endpoint
					address ="soapService"
					binding ="wsHttpBinding"
					contract="PPMRK_PLMService.IPLMService"
          bindingConfiguration="tcpWithMessageSecurity"
					/>
<endpoint
					address ="soapUIService"
					binding ="wsHttpBinding"
					contract="PPMRK_PLMService.IPLMService"
          bindingConfiguration="tcpWithMessageNoSecurity"
					/>
2. Then Add the new binding configuration for the new endpoint.
<binding name="tcpWithMessageSecurity">
          <security mode="Message">
            <message clientCredentialType="UserName" />
          </security>
        </binding>
        <binding name="tcpWithMessageNoSecurity">
          <security mode="None">
            <transport clientCredentialType="None" />
            <message establishSecurityContext="false" />
          </security>
        </binding>
3. Fix SoapUI 
Now you can configure SoapUI, bind to the wsdl as you would. In the request, click the WS-A tab at the bottom and check the add default wsa:action and Add default wsa:To boxes. 
4. Profit!

UserName ws-Security Header with WCF

If you work with web services for customers to call, security has to be a top priority.  The Soap protocol has something called the ws-Security Header which you can read more about here http://msdn.microsoft.com/en-us/library/ms977327.aspx.  Now WCF can be configured about infinite different ways.  What I wanted to do was just make a simple Username Password passed in the header of the service call over https.  From what I have gathered, this is the best way to do it in WCF.

1. Configure your web.config. 

<?xml version="1.0"?>
<configuration>

  <appSettings>
    <add key="aspnet:UseTaskFriendlySynchronizationContext" value="true" />
  </appSettings>
  <system.web>
    <compilation debug="true" targetFramework="4.5" />
    <httpRuntime targetFramework="4.5"/>
  </system.web>
  <system.serviceModel>
    <services>
      <service name="PPMRK_PLMService.PLMService" behaviorConfiguration="CustomValidator">
        <endpoint
					address ="soapService"
					binding ="wsHttpBinding"
					contract="PPMRK_PLMService.IPLMService"
          bindingConfiguration="tcpWithMessageSecurity"
					/><baseAddress><add baseAddress="http://localhost/PLMService" /></baseAddress>
      </service>
    </services>
    <bindings>
      <wsHttpBinding>
        <binding name="tcpWithMessageSecurity">
          <security mode="Message">
            <message clientCredentialType="UserName" />
          </security>
        </binding>
      </wsHttpBinding>
    </bindings>
    <behaviors>
      <serviceBehaviors>
        <behavior name="CustomValidator">
          <serviceMetadata httpGetEnabled="true">
            <serviceCredentials>
            <userNameAuthentication
              userNamePasswordValidationMode="Custom" customUserNamePasswordValidatorType="PPMRK_PLMService.Authentication.CustomUserNameValidator, PPMRK_PLMService"/>
            <serviceCertificate
              findValue="localhost"
              x509FindType="FindBySubjectName"
              storeLocation="LocalMachine"
              storeName="My" />
          </serviceCredentials>
        </behavior>
          <behavior>
          <!-- To avoid disclosing metadata information, set the values below to false before deployment -->
          <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true"/>
          <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
          <serviceDebug includeExceptionDetailInFaults="false"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <protocolMapping>
        <add binding="basicHttpsBinding" scheme="https" />
    </protocolMapping>    
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="false" />
  </system.serviceModel>
  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true"/>
    <!--
        To browse web app root directory during debugging, set the value below to true.
        Set to false before deployment to avoid disclosing web app folder information.
      -->
    <directoryBrowse enabled="true"/>
  </system.webServer>

</configuration>

You can see this isn't a stock config file, well hopefully you can.  Inside the system.serviceModel tags, I added services tags, a new behavior and bindings to get this to work.  The service is declared explicitly to bind to the new behavior called CustomValidator as well as declare the endpoint with a tcpWithMessageSecurity bindingConfiguration. 

Next, the bindings gets a new netTcpBinding what has a message security mode with a clientCredentialType of Username. This will make it use a username password header type.

Finally, The new behavior added I call CustomValidator binds the authentication of the username password to a custom class I wrote (below) and a certificate I have installed (how to below further).

2. Create the Authentication Class

public class CustomUserNameValidator : UserNamePasswordValidator
    {

        public override void Validate(string userName, string password)
        {

            if (string.IsNullOrEmpty(userName) || string.IsNullOrEmpty(password))

                throw new SecurityTokenException("Username and password required");

            if (true)//validate

                throw new FaultException(string.Format("Wrong username ({0}) or password ", userName));

        }

    }

This class simply implements the UserNamePasswordValidator (System.IdentityModel.Selectors) class's Validate function.  Not hard.

3. Create Cert (for Dev to Start)

What you need to do here is fire up the Visual Studios Command Prompt (http://msdn.microsoft.com/en-us/library/ms229859(v=vs.110).aspx) and use the makecert function to create a cert to test with.  (http://msdn.microsoft.com/en-us/library/ms733813.aspx).  I ran my make cert like this.

makecert -sv C:\temp\Sign_Root2.pvk -cy authority -r C:\Temp\Sign_Root2.cer -a sha1 -n "CN=Dev Cert Authority2" -ss my -sr localmachine

I had to run this as administrator, or I got errors, so keep that in mind. 

4. Install the Dev Cert

This can easily be done by right clicking and installing the .cer file you created.  Note my cert goes to local machine.  You may need to run certmgr.msc or mmc as a local admin and give rights to the cert to your main account for it to work correctly. 

5. You should be able to run your service and see the wsdl, it now has ws-Security Header stuff installed.