August 2007 Archives

Whitehorses, a consulting organization specializing in innovative software development based on Oracle and Java products, asked me to write a brief article on developing secure web applications.

The article is available here. Unfortunately, it is in Dutch :)

FBI Wire tapping systems

| No Comments | No TrackBacks

Liquidmatrix Security Digest pointed me to an article in Wired about the FBI's wiretapping cabilities.

DCSNet is a suite of software that collects, sifts and stores phone numbers, phone calls and text messages. The system directly connects FBI wiretapping outposts around the country to a far-reaching private communications network.

[...]

Today, most carriers maintain their own central hub, called a "mediation switch," that's networked to all the individual switches owned by that carrier, according to the FBI. The FBI's DCS software links to those mediation switches over the internet, likely using an encrypted VPN. Some carriers run the mediation switch themselves, while others pay companies like VeriSign to handle the whole wiretapping process for them.

Now; reading this article reminded me of a very good writeup of the Athens Affair in IEEE Spectrum of July 2007.

What whould happen if this large-scale infrastructure somehow got compromised? Would the intruder have full access to all our phone calls in real time? What kind of controls are in place to prevent such mis-use? What kind of controls are in place to detect breaches, and how would intrusions be reacted to? If the DCS system is indeed linked to the internet, an attack vector must exist. Much is known already, and more information is expected to become public.

I'll be interested in seeing more about this.

VMWare Server on Ubuntu Linux

Note to self:

when installing a new version of VMWare Server, make sure to grab the vmware-any-any-update113 patch from http://knihovny.cvut.cz/ftp/pub/vmware/. Without it, compilation of the kernel module vmmon will fail. Also, make sure that the kernel headers are there: apt-get install linux-headers-`uname -r` (hint found on http://www.vmware.com/community/thread.jspa?messageID=580275)

Update: To prevent the log files from flooding with useless information about your real-time clock (VMWare rtc: lost some interrupts), make sure that /etc/vmware/config contains the line host.useFastClock = FALSE. (hint found on http://chxo.com/be2/20060821_3333.html) (yes Anton, there is such a thing as useless information in log files :)

Christofer Hoff pointed me to a publication in the Harvard Business Review. The article describes a case study in which a (fictional) company suffers a data breach. The case study really revolves around the proper organizational response and boils down to one question: going public or not.

"Brett, we have a problem. There might be a data breach." Laurie, a tough but polished former Chicago police detective, had been responsible for security at Flayton's for almost three years. She had an impressive record of reducing store thefts while building productive relationships with local schools, community groups, and law enforcement.

"What kind of data breach?" Brett asked. His tone was calm, as always, yet he scanned the lounge to make sure that no one could overhear.

"I'm still not sure," Laurie admitted. "I was contacted by Union Century Bank. They regularly examine their fraudulent accounts for patterns, and we've shown up as a common point of purchase for an above-average number of bad cards. They're getting me more information, but I thought you'd want to know right away. It could be nothing--or it could be significant."

The choice of words, as well as the writing style used in the article makes it a very pleasant and informative read. Put yourself in the hot-seat: disclosure or not? At what price? What is it that your customers expect from you, and how will the respond when that expectation is not met? What about the your other stakeholders?

My first gut-feeling: glad I'm not in public relations :)

In a situation like this, an organization should admit when it is at fault and work hard to regain the trust that its stakeholders have put in it. Keeping it quiet will only work against you and should be avoided. Some information should be made public, but in a joint effort with the banks and (possibly) law enforcement.

Does anybody else strike this as mildly intriguing?

Title: IT Security Architecture Manager Needed

Company: TJX Companies

Location: Framingham, MA

Skills: Very strong technical security background in both the mainframe and distributed environments.

Term: Full Time

Pay: DOE

Length: Full Time

Intrusion report at fark.com

| No Comments | No TrackBacks

Chuvakin pointed me in the direction of this article about an alleged hack of fark.com's systems.

The story makes a very interested read, mostly because of the level of detail that it contains.

Microsoft Security Response Center (MSRC)

One of the things that makes blogging interesting is the ability to leave feedback, such as comments or trackbacks. Microsoft's Security Response Center (MSRC) also has a "blog".

Please note the use of double quotes around the word blog. While I have been subscribed to the "blog" for a long time, it did not strike me until recently that it does not allow any feedback to be submitted, even after signing up for an account.

One might wonder why.

As far as content goes: the "blog" seems to house nothing more that carefully screened posts that offer little or no insight into the workings of the center, nor does it provide much original thought.

With a few exceptions, the blog posts a summary on patch Tuesday and an announcement the Thursday before. Very rarely does it go much further than that.

Of all the vendor blogs that I subscribed to, this must be the least useful one.

Anton Chuvakin pointed me to this blog article on a Gartner Group Security & Risk Management blog. The article lists a number of common pitfalls in security. Interestingly, the following points are among those made by the analyst who wrote the entry:

5. Most likely to result in less security: Compliance efforts

6. Most likely to result in more compliance:
Security efforts

Much of the focus that the information security field has at the moment is directly caused by C-level requirements to be "in control" and "compliant". Gartner Group makes the point (which I have tried to convey for a long time to many of my own clients) that compliance will not lead to security, but that increasing security will lead to compliance.

In the whole compliance debate, goal and means have been confused for too long. The goal of the whole compliance process is to protect stateholders interests, and some of the means that can be used to achieve that, are security related. Compliance is not the goal. Protecting stakeholder's interests is.

Even port scanners are boring...

After setting up a virtual private server with the people at rimuhosting.com, I was curious to see what a box like that has to deal with as far as port scanners go.

To find out, I ran tcpdump for 24 hours, without any filtering:

sudo tcpdump -nUi eth0 -w tcpdump.log 

24 hours later, I terminated the tcpdump process and filtered out some of the known traffic, such as my machine at home, and some others, and I only looked for 'SYN' packets.

/usr/sbin/tcpdump -nr tcpdump.log -l '(not host xx.xx.210.33) and \
  (not host xx.xs4all.nl) and (tcp[tcpflags] & tcp-syn != 0) and \
  (tcp[tcpflags] & tcp-ack = 0)' |egrep -o '([0-9.]+ > [0-9.]+)' > tcpdump.txt

That resulted in a file of only 2159 lines. Those lines were formatted as so:

xx.yy.196.35065 > aa.bb.224.141.80

This gave me the originating IP, with its TCP port number, as well as the destination IP and port number.

First step: extract all the destination ports:

awk '{print $3}' tcpdump.tmp |cut -d. -f5 |sort -n |uniq > ports.txt

With all destination ports safely in a file, I can now summarize:

for i in $(cat ports.txt); \
do \
  n=$(egrep "\.$i\$" tcpdump.tmp |wc -l); \
  echo "$i=$n" ; \
done |sort -t = -k 2 -rn

Which leads to

80=1731
25=64
2967=62
5900=51
1433=50
22=36
135=29
139=24
1080=22
445=19
443=13
8080=7
3306=4
21=4
10000=4
8395=3
4899=3
6900=2
6348=1
43=1
2968=1
2222=1
113=1

Nothing surprising here; port 80 is where my web server lives and port 25 is my (outgoing!) mail. Port 5900 is VNC, which is know of its many issues. 1433 is an old friend (SQL Server) and is still scanned a lot. From this point on. it quickly becomes less interesting: 135 and 139 are Microsoft Windows Ports, 1080 is Socks (looking for an open proxy?) 445 is more Microsoft; 443 is HTTP over SSL (looking for proxies?), 8080 is more proxy searches, 3306 is MySQL, 21 is FTP (as if :), The rest of the ports doesn't really count.

Let's soon in on 2967 and on 22. Starting with the latter one, it is well known that SSH bruteforce scanning is very popular at the moment. My 36 scans originated from 9 hosts

awk '/\.22$/{print $1}' tcpdump.tmp |sed -r 's/\.[0-9]+$//g' |sort |uniq
x.214.29.173 (Japan)
x.215.190.134 (Taiwan)
x.50.197.234 (USA)
x.108.231.56 (China) 
x.130.189.223 (China)
x.143.10.5 (China)
x.78.145.92 (USA)
x.149.219.232 (UK)
x.42.7.23 (Romania)

I have set up IP port filtering to only allow incoming SSH connections from a few trusted IP addresses, so these attempts all died before they even began.

The second batch is for port 2967, which is the only one that I did not know.

awk '/\.2967$/{print $1}' tcpdump.tmp |sed -r 's/\.[0-9]+$//g' |sort |uniq
x.22.79.222 (China)
x.139.58.112 (China)
x.241.178.220 (China)
x.142.132.9 (USA)
x.19.186.202 (USA)
x.19.188.118 (USA)
x.231.2.33 (USA)
x.233.213.29 (USA)
x.243.227.28 (USA)
x.30.98.53 (USA)

THis one is more interesting; while only two of the SSH scans originated from the USA, 7 out of 10 scans for port 2967 come from the USA. Port 2967 is used by Symantec System Center, and DShield has an interesting graph about it too. The remark that especially colleges are targeted is not something that I see from my brief analysis; not one of the originating IP addresses belonged to a school.

All in all, it seems like there isn't really all that much going on in portscanning-land.

Maybe I'll try this again some day on my home box.

Storm botnet fighting back

InfoSecNews put an article on the wire today about the Storm Botnet getting more aggressive.

With schools starting within a few weeks in many different parts of the Northern hemisphere, many network administrators are running active network scans to look for traces of malware. Often, malware is easy to detect because it keeps a backdoor listening. The presence of an open network port on a host is often the first hint that something is afoot and almost always requires further analysis.

The ISN article outlines that the malware detects that it is being scanned, and instructs (part of) the vast botnet to attack the host from which the scans originated.

More often than not, these hosts are regular workstations used by administrators and, even if the intermediary infrastructure (firewalls, routers, switches, etc.) can handle the DDoS-volume, those workstations more than likely cannot. Effectively, such retaliation attacks will make it much more difficult for network administrators to find out what is happening on their network.

What does this mean?

Does it mean that admins can no longer actively scan for traces of malware on their network, for fear of retaliation?

Possibly.

Fortunately, there is still the possibility to listen passively. Using tools, such as netflow, it is still possible to detect suspicious network traffic patterns. The benefit of using netflow, or other forms of passive network monitoring, is that it is (near) impossible to detect by an attacker.

While an attacker can still attempt to launch a DDoS-attack against the school, it will be harder to find out what the exact target is.

In the mean while, it seems like the botnets are getting more aggressive about fighting back against detection and cleanup. I wonder where this will take us next.

Web site maintenance

| No Comments | No TrackBacks

I have recently upgraded the back-end of this site to MovableType. Somehow, I managed to play around with my style file a little too much, and as a result, some pages are now really ugly. Especially entry details need a lot of work. Once I figure out what I did wrong, I hope I'll be able to fix it. Until then, please choose content over appearance :)

I received a good phishing email today; so good that I actually had to devote some brain cycles to make sure that it was indeed a phish. Being involved with computer security incident response myself, I write a nice report about the message, sign it, and send it off to abuse@ebay.com.

To my surprise, it bounces back with a message stating that "This email address is no longer in service".

Wft?!

I would think it is in the best interest in any organization to have a working abuse address; let alone a company that is pretty much constantly under attack. How dumb can you be to discontinue an RFC-recommended email address that anyone who knows a little bit about incident response would use?

I have been making a point in several places that application security is increasingly important. Attackers are increasinly moving higher in the OSI stack. This observation is not something that I have come up with myself, nor is it something that is too terribly shocking.

However, I do feel that many security professionals are aware of the problem, but despite Y2K and Euro conversion issues, that they don't really fully understand the scale and the potential impact of deficient code.

This month's Microsoft patches have been released. Of the 9 patches that are released, four are directly caused by input validation errors (MS07-044, MS07-046, MS07-048, and MS07-050).

Three of the four vulnerabilities are rated as critical and one is rated as important. Input validation errors are among the easiest to prevent by adopting proper coding practices and/or by (automated) code review.

Also interesting to note is that the fortress of virtualization is beginning to crumble. This month's patches include MS07-49, which allows an attacker to exploit a vulnerability in virtual PC to run arbitrary code on the host OS. It is easily conceived that an attacker, by compromising a host OS, can hop onto other guest OS'ses and by doing so, easily compromise a large number of hosts.

I want to go too!

| No Comments | No TrackBacks

Really, I do!

I was recently asked to give a 10-minute lecture on three topics relating to information security, of my own choosing. The article was written from the perspective of an information security officer in academia. Here it goes:

Organizations rely increasingly on information as a primary asset. Educational institutions have acknowledged this very early on, and while research and education benefits immensely from an open environment, universities were among the first organizations developing and adopting information security.

The field of Information Security is defined by three principles: Availability, Integrity and Confidentiality. By availability, we mean that any information system, including the information and data that is contained within them, is available to their intended users, whenever they need it to fulfill their assigned tasks. The integrity principle aims to ensure that data that is correct and complete. Confidentiality ensures that only duly authorized individuals can access information.

To assure the appropriate levels of Availability, Integrity and Confidentiality, organizations need to be aware of the risks that are threatening their information systems. Many risks can be easily reduced by assessing the vulnerabilities of information systems and taking appropriate measures. Information Security best practices have been captured in an international standard, known as ISO 17799. This standard describes a large number of controls that organizations may adopt to protect their information assets.

Given the time that we have today, I will speak a little more about three areas of information security: user awareness, monitoring and incident response, and regulatory compliance.

A phrase that is commonly heard when information security practitioners communicate, is that "perimeters are fading". That expression immediately betrays that information systems security originated as a technical discipline. With systems being accessed from increasingly mobile users, it becomes hard to define hard perimeters of a network. Users connect using wireless equipment, when they are traveling to conferences and are staying in hotels, or when they work from home. The era when 100% of an organization's network was fully under its control are over.

However, this perspective is flawed. An organization consists of a collection of individuals. Each of those individuals has a personal agenda, which always takes the front seat. Second to that comes their professional agenda. Social scientists, who have studied organizational dynamics acknowledge this, and even make it a fundamental principle in their theories. I believe that it is time that information security professionals also acknowledge this, and adopt the point of view that any organization, and certainly a university, should consider that the organization's perimeter is made up of its users: staff, faculty, students, third-parties, partners, etc.

When this view is adopted, it is immediately clear that information security controls that need to be in place must be focused on those users. Users must come to acknowledge that information is important, and that it is worth protecting. To do so, they must not be told what they can do, and what they cannot do, but they should be made to understand why controls are in place and why they directly benefit from having them. A good awareness program will lead to users willingly adopting information security controls.

The added value that an effective awareness program can deliver was recently illustrated by an article in the Wall Street Journal, which attracted much attention. The title of the article is: "Ten Things Your IT Department Won't Tell You". In the article, the author outlines how to circumvent file transfer restrictions using online services, such as YouSendIt, how to use software that the company won't let its users download or install, or how to store work files online. There are many more so-called "tricks" that the author lists. By doing so, the Wall Street Journal directly undermined much of the awareness work that was done in many organizations. "See, if the Wall Street Journal publishes it, it cannot be that bad, right?". What the article fails to mention is why an organization has those controls in place, how organizations monitor compliancy with the controls, and what the repercussions are for employees who willing bypass security controls.

As mentioned earlier, availability, integrity and confidentiality define the field of information security. In addition to implementing technical controls that are aimed at technical systems, organization should devote at least as many resources at implementing organizational controls, such as awareness programs, policies, and procedures.

The second topic that I want to touch on is incident response. Public safety officials acknowledge that, despite all the prevention efforts that are made, fires will break out, natural disasters will happens, shops will be burgled, and cars will be involved in accidents. This is why a large part of the public safety efforts are devoted to reactive capabilities of police officers, firefighters, and paramedics. Like their counterparts in the physical world, information security professionals will never be able to provide security for 100%.

Some measures will fail, and attack vectors will be developed for which no existing defenses are in place. Organizations, must be able to detect information security threats, and the incidents that follow from them, and develop the capability to respond to them.

Because of the increasingly interconnected nature of information systems, organizations cannot address these threats alone. Because of this, there are many ways to collaborate with other institutions, such as law enforcement agencies, US-CERT, the forum for incident response and security teams (FIRST), NSP-SEC, the University Security Operations Group, etc.

Universities are popular targets by attackers. Not so much because of the value of the information that can be obtained by them, although there is certainly much information of high-value in educational institutions, but much more because of the relatively high network bandwidth that University's have available, and the relative level of anonymity that can be obtained on university networks.

Computers in dormitories, in computer labs, and even Faculty PC's are very often infected with malware that turns them into so-called "zombies" or "bots". These bots are subsequently used to send spam, collect personally identifiable information, spread copyrighted materials, or to participate in large scale denial-of-service attacks. Implementation and execution of effective monitoring and response policies reduces the impact that compromises may have on networks, and also reduces the chance of undesirable media attention.

Universities also harbor some of the worlds most intelligent young people, who are driven by a constant drive to explore and experiment. While not that common, some students will not possess the appropriate level of ethics, that will prevent them from exploring the University network. Grade registration, tuition payments, admission information, etc., are all likely targets and an appropriate defensive, detective and reactive capability should be developed before incidents actually take place.

Whatever the controls will be that are put in place, they should be mandated, sponsored and practiced by senior management positions. While handling security incidents, it should be absolutely clear what the responsibilities of the responder are, but also by what authority his or her actions are mandated.

To assure that availability, integrity and confidentiality of information is kept at acceptable levels, even in times when normal operations are interrupted because of natural events, malicious attacks, or technical interruptions, organizations need to develop an incident response capability.

To develop this capability in a cost-effective and efficient way, cooperation with existing bodies should be established.

The final topic that I want to touch upon is regulatory compliance. Legislation and industry practices are putting increasingly stringent demands on the way that an organization structures its processes, protects its resources, and makes the controls that are put in place auditable. Legislation such as Sarbanes-Oxley for publicly traded companies, HIPAA in healthcare, or industry standards such as the Payment Card Industry's Data Security Standard are constantly developing and expanding.

When an organization, such as a University, is unable to meet these requirements, it will be faced with problems in attracting funding from many sources, including the federal government. It will also be faced with fines, or even with the inability to accept credit card payments. In some extreme situations, executive level employees might even face criminal charges.

A well-managed information security program can address regulatory compliance and prove to stakeholders that an organization practices due diligence and due care.

To summarize: information is the primary asset of many organizations, and especially of educational institutions. Protecting that information in terms of availability, integrity and confidentiality is the responsibility of an information security program. In addition to implementing technical controls, organizations must realize that information serves the organization's stakeholders and not the other way around. This is why effective organizational controls, such as awareness programs, well-written and though-out policies and procedures are at least as important as technical controls.

Despite all measures that an organization takes, security incidents will happen. When they happen, organizations must be aware of them, and have developed the proper capabilities to respond to them.

Regulatory compliance, originating from legislation, as well as from industry requirements, will put increasingly heavy requirements on organizations. Having a well-managed security program will prove to stakeholders that an organization practices due care and due diligence and will make compliancy easier to achieve. •

Michael Farnum quits...

| 1 Comment

Michael Farnum wrote a post on his blog, in which he referred to another post that said:

  • Existing defenses are absolutely ineffective against current attacks. I am struggling to describe the importance of this insight. It does not matter if you are fully patched, "properly configured," not running Javascript, or adopting any number of other current defensive stratgies if you use a Web browser that renders modern rich content. Almost none of the techniques described in the Black Hat talks relies upon exploiting vulnerable software. Almost all of them abuse inherent functionality for malicious reasons.
  • Detecting current attacks in "real time" is increasingly difficult, if not impossible. Even if you assume attacks are not obscured by encryption, recognizing and understanding the variety of Web-based attacks shown at Black Hat is almost a lost cause. There is basically no way for defenders to address the expanse of the attack surface exposed by "rich Internet applications" and frameworks. I realized that the "rich" in "RIA" refers to the money intruders will make by exploiting Web clients.
  • The average Web developer and security professional will never be able to counter these attacks. Intruders are so far ahead of the defenders with respect to tools and techniques that it is simply not possible to prevent the attacks I saw at Black Hat. This statement will probably offend many people but it's time to face the truth. There is no way to get "ahead of the threat" here.

Michael states that he now officiall depressed.

While I mostly agree with the observations, I do not agree with his conclusion; I was at the annual FIRST conference last year, and I attended a session about the malware marketplace. If you see the amount of (criminal) information that is being traded in underground networks, and the prices at which these trades are made, you know that as an information security professional, you should have your technical and administrative controls in place, but you should never have the illusion that you are fully protected. It is too easy for a determined attacker to bypass the defenses that you have in place by finding new attack vectors that you have not thought of before. A typical example is the continuous information leakage that happens through peer-to-peer file-sharing networks, such as eDonkey, and company. The fact that organization perimeters are fading is not new.

It is my personal opinion that effective information security needs technical measures, but that a /real/ awareness in the user constituency is crucial to success. An organization's perimeter must include the organization's constituents The Wall Street Journal article of July 30, 2007 illustrates this.

Another observation is that things will go wrong and that properly developed incident response capabilities are vital; in the public space we also acknowledge this by having ambulances and firetrucks manned at all times; despite all the prevention that we do, things will go wrong.