What I learned last week at work #3

In a 3 day week, I only managed to learn how to get distinct IP addresses from log file.

How to get distinct IP addresses from log file

For a customer of ours, I had to screen two years of log files and find distinct IP addresses for certain criteria. You could check those log files by hand. Sure, it would take a month or two, but it can be done. However, if you are not keen of spending your day looking at log files line by line, here is what you can do:

  1. You can grep log files for specified criteria:
  2. Then you can parse results to get all IP addresses:
  3. You can then use awk, to print them on separate lines:
  4. And again use awk, to print only distinct ones:
  5. Optionally, you can store output to file:

Ideally, you want to run this in one command:

There you have it! File _ip_addresses.log now contains only distinct IP addresses.

I am pretty sure, it can be done differently. You can leave your solution in comments below.

What I learned last week at work #2

It’s been a quiet week at work. Fixing a bug here and there, implementing minor features, writing some documentation etc etc. Hence, this weeks findings are not programming related.

Without further ado, here is what I learned last week:

  • Windows 10 app restart on unexpected shutdown (or after update restart) cannot be disabled;
  • Solving ‘PkgMgr.exe is deprecated’ error.

Now to details.

 

Windows 10 app restart on unexpected shutdown cannot be disabled

Since Fall Creators update Windows 10 gained an interesting feature. Much like OS X, it restores your applications upon unexpected shutdown or maintenance restarts. Now, I bet this feature sounds great on paper and I bet it is perfect for your everyday user. However, the feature is totally useless and annoying to anyone, doing something more with his/hers computer, besides browsing the internet and watching occasional X rated movie.

Imagine this. At the point of maintenance restart (updates have finished installing), I have 7 Visual Studios 2012 in administrator mode, 5 Visual Studios 2010 (again in administrator mode), 6 Microsoft SQL Management studios, a Notepad++, Outlook, 3 Word documents and 5 Excel worksheets open. I am not even going to count remote desktop sessions and other minor software windows. Now, computer does reboot, it comes back and and I am presented with login prompt. After typing my password 3 times (seriously, I need another password), OS starts loading all windows mentioned above. Except, it opens all Visual studios in normal mode and without opened solutions (thanks for that, btw). Same goes for MS SQL Management Studios. It opens 6 instances, not one having an active connection or at least a correct SQL instance selected. Useless and annoying.

To top it all off, apparently, this feature cannot be turned off and no upgrade to make this available is scheduled to this point.

Solving ‘PkgMgr.exe is deprecated’ error

After a server came crashing, we had to set up a new one. After completed install of server roles and features and our applications, I tried running some of them and got Service unavailable error. I tried to register .NET by issuing

command. This returned another error PkgMgr.exe is deprecated. Quick googling found this page, that explains the cause for the error is missing ASP.NET installation. I went back to server installation and selected ASP.NET 3.5. That solved the problem.

What I learned last week at work

I am a firm believer of a fact, that if you are not learning anything new at your work, it is time to move out of that comfort zone, pack your bags and find a gig where you will. Lately, my work shifted and consists of 99% maintenance grunt work and 1% of actual new development. In that kind of situation, a person can easily forget, that despite chewing the dog food, there is an occasional pickle here and there. So, I created this series. To remind myself, that I am still learning something new and to, hopefully, provide some extra value to whomever stumbles to this place.

So, these are the things I learned in past week:

  1. The verb INTO is not necessary when running INSERT SQL statements on Microsoft SQL Server;
  2. Direct cast of column value of System.Data.DataRow object in .NET 1.1 does not work anymore on Windows Server 2012 and Windows 10;
  3. How to compare strings with fault tolerance;

Now to details.

 

The verb INTO is not necessary when running INSERT SQL statements on Microsoft SQL Server

Debugging for some odd mishap, I have located the following piece of code:

According to SQL standard, verb insert should be followed by verb into. Except it wasn’t. I thought that this has got to be some obsolete code that no-one uses. I’ve checked references and found a few. So that wasn’t it. The code obviously worked, as it exists since 2012. So what the hell?! Well, it turns out, that even though the verb into is mandatory by standard, most implementations (Microsoft SQL server included) ignore this and keep it as optional. I am definitely not adopting this, but it certainly is interesting.

 

Direct cast of System.Data.DataRow column value in .NET 1.1 does not work anymore on Windows Server 2012 and Windows 10

Yes, I know. Microsoft stopped supporting .NET 1.1 framework with Windows 7. Still, we have some projects that run (or more accurately ran) properly even on newer Windows OS. Except that with every update to Windows 10 and Server 2012 it is more and more obvious that .NET 1.1 is getting pushed out.

The latest thing was an InvalidCastException when executing this statement:

where row is of type System.Data.DataRow. One would think that value is not integer, but in this case it was 103, which by my books, is an integer. Interestingly enough, this works:

Go figure.

 

How to compare strings with fault tolerance

In one of our projects, searching by peoples name and surname just wasn’t good enough. Spelling mistakes and different characters in place for unicode ones were supposed to be taken into account.

After 5 minutes of “googling”, I found a StackOverflow answer that suggested using Damerau-Levenshtein distance algorithm. Levenshtein’s distance algorithm provides a way to calculate number of edits that need to be done on one string to get another. Damerau-Levenshtein algorithm is an upgrade that also allows characters to be transposed.

However, this is just the first step. Algorithm provides you with a number of edits. To use it, you still need to define a threshold of how many mistakes will you allow. Fixed values are just not good, if your string length varies. So, I used half of the length of either search query or provided value. It works like a charm.

Quick tip: Optimizing repeating try-catch-finally statement

Lately, I’ve started noticing a pattern in data layer of one of our projects at work. The pattern looks like this:

This repeats itself in just about every data layer method. Lines and lines of useless, repeating code for which I am also to take a lot of blame. So I thought: “There must be a better way than this.”

And there is. I created this method in data layer base class:

This enables me to now change every data layer method to look like this:

This solution has a small issue though. If you are doing insert or update, you might not want to return anything. As you cannot return void, just define returning type to be object and return null. I am prepared to live with this.

 

Failed to load resources from file. Please check setup

Not so long ago an application written in .NET 1.1 started to pop this error up and about. Funniest thing though, only Windows 10 clients with Creators update installed were affected. Now, we could argue, why there is still an application written in .NET 1.1 and running, but that could be a lengthy debate in which I really don’t want to go into right now. Or ever.

Anyway. The error, as descriptive that it is, means only one thing: somewhere in your code, there is a StackOverflowException. In case you are wondering, no, event logger won’t detect a thing. After much trial and error, I have narrowed the problem down to this chunk of code:

Method GetValueEx returns a response of type object. In this particular case, it should have been a string, but as there are no hits in the database, it returns null. So, basically, the line 3 of method GetValue should have thrown a NullReferenceException, which catch statement should have caught. Except it doesn’t.

I don’t have enough information to explain all details, but on Windows 10 Creators update line 3 throws StackOverflowException, which is for some odd reason not handled by try-catch block. And this causes “Failed to load resources from file. Please check setup” error.

Knowing this, I modified my code to:

Needless to say, the fix works without a glitch. Being a good Samaritan, I have also posted the answer to this StackOverflow question.

TimeLoggerLive early-bird pre-order

There has been a lot said and written about how people should log time they spend on tasks. Some claim you should log only the time you actually worked on a task, some that you should log all time, including intrusions, lunch breaks etc. And from the project standpoint, I agree with later option. However, when it is you, who needs to track where your time went, you are faced up with a difficult task.

I guess you could use time logging features of your project management tool, but that is usually tedious and time consuming. Not to mention unpractical.

You could use one of the thousand apps that are out there, that require you to just press start button when you start timing the task, and stop button, when you stop doing it. But these usually come up with results in form of 2 hours and 33 minutes when you really wanted to log 2 hours and 30 minutes. This leads to editing and even more time lost. Also, all applications I have seen and tested, require you to enter tasks first, which is in my books double work. Specially when we use project management tool.

Personally, I use pen and paper. Archaic and non-environment friendly. It works, but it has a serious issue. In my line of work, I do a lot of context switching and at the end of the day, I spend some time just summing up time spent for tasks. It is not particularly time consuming, but it is tedious and error prone.

So, I have this idea of a time logging web application, that would be as simple as logging time on paper. Just start time, end time and description of your work. Description would have type-ahead of already entered descriptions, enabling easier summation of your daily time consumption. And you would be able to get nice condensed report for each day, week and month.

In future editions, integration with popular project management tools like JIRA, Trello, Trac is imminent. But for now, I aim for simple and get things done principle.

With all said and written, today, I can proudly announce TimeLoggerLive early-bird pre-order is available. As an early bird, you are entitled to:

  • minimum 30% lower subscription price for first 3 years (1st year at 80% off),
  • access to all development and future versions of the application,
  • hassle-free, any-time money back guarantee,
  • personalized and friendly support.

 

And the best part is, even if you decide to leave TimeLoggerLive, we will keep your data (unless otherwise requested) in read-only form, available to you online, if you ever need it again.

For companies, the product will also feature creation of teams and overview of their logged activities, bundling tasks into meaningful projects and user management.

TimeLoggerLive is currently in development and is expected to go live November 1st, 2017. I expect first beta to be done by August 1st, 2017 at the latest. By registering today, you will help TimeLoggerLive become awesome. And personally, I look forward to have you as a customer.

Minimal Web API

For a task at work, I needed a mock WebApi for mimicking behavior of 3rd party API, that is not accessible from my machine. Sure enough I proceeded using line of least possible resistance and selected ASP.NET MVC Web Application template and WebAPI and ended up with bloated project that scared the life out of me when I just looked at it. Seriously, Microsoft, you couldn’t do a simpler template? I am pretty sure, that having all sorts MVC thingies installed helps, although, I am not completely sold on EntityFramework that sneaks in as well. But I digress.

Without further ado, I present you minimal WebAPI. A project that contains only what it needs to, to behave as WebAPI and nothing more. Below, I will lead you step by step through the process of creating one yourself, but you can also clone a repository from GitHub. Be careful though, as solution only supports .NET 4.6.x.
[Read the rest of this entry…]

Custom software – Introduction

Custom software is software that is usually done for one client, one environment and limited user base. It’s specific use case means, it won’t be sold more than once, which translates to high enough price to cover development costs. And then some.

So why do companies decide to order and pay for custom software over and over again? Custom software has one big advantage over regular off-the-shelf one. It is adaptable to whims and fancies of a customer. Hence, the name custom. Now, here is a public secret. Companies, specially wealthy ones, dig adaptable. You see, where smaller companies don’t have (many) defined processes, big companies are set on tightly defined processes, usually certified by ISO 9001 or similar standard. Off-the-shelf solution is just not an option, as changing a process would cost a lot of money, time and resources.

As a developer with a career in building custom applications, I have done and seen my share of mistakes and fallen into several pitfalls. This series is here to help you (and also future me) avoid them as best as possible.

In part I of the series, I will be blabbering about project architecture and discover last year snow in monolith vs microservices debate. We will cover how to start, how to continue and how to take care of the accumulated tech debt.

Part II will be focused entirely on how to integrate your software with third party software. There is always something to integrate with in custom software business and it is better to be prepared.

Part III will touch the topic of what happens when your custom software gets resold to another and another and another customer. Main focus will be on how to handle new requirements and hopefully not mess up existing and future deployments.

Last but not least, a disclaimer. This series is made of findings I gathered in my career of building custom software. As many things in software, it definitely does not present the only right way. In some cases you might also find it completely wrong. Good. Let me know.

Windows 10 changes Slovenian locale (at last)

If you are a proud developer of software sold internationally or in Slovenia, you might pay a little attention to this. It has not been noted anywhere of importance and I discovered it by chance when one of the programs I am working on crashed.

Versions older than Windows 10 had the following Slovenian date and time format: “d.MM.yyyy”. As far as this can be understood from computing point of view, it is in conflict with Slovenian language rules.

So, in Windows 10, someone made a bold move and fixed said format to: “d. MM. yyyy”. Finally, one might say. Except, every possible application, relying on previous false date format, will now crash. This is specially true in C++ Ole objects, where calling COleDateTime::ParseDateTime with OS set language now fails, if string representation of date is in “d.MM.yyyy” format and OS primary language is Slovenian. Let me state here, that .NET handles date formatting without problems.

If you are targeting Windows 10 clients only, this is not a problem, per-se. You just need to make sure that there are spaces after each dot in date time string representation. If you are targeting multiple OS versions, you are cooked as older Windows, do not recognize new date time format.

I am still searching for least painful workaround, that would work on any Windows OS client. If anyone has an idea, you are more than welcome to share.

Know thy tools

A few years ago, I wrote a post on why I don’t trust 3rd party APIs. It has never been more true than today (in days of NuGet, npm and other package managers) when adding an API is a matter of seconds. Seriously, you don’t even have to break a sweat. Package managers and web searchers got you covered. We live in the days where even “evil” Microsoft decided to move to the good side of the force and went open-source. Right? APIs get updated more promptly than ever and they are available for just about anything. And code reuse saves you a lot of time. So why am I still skeptic about it?

[Read the rest of this entry…]