Logging of network access

It is vital within the university environment that the source of any abusive or malicious network traffic can easily be traced and isolated. Depending on the nature of the problem, it may be necessary to determine either the user or the computer responsible for particular traffic at a particular time. OxCERT will expect colleges and departments to be able to trace either upon request.

Reasons for requiring traceability include, but are not limited to, the following:

malicious network traffic (scanning, sending spam, communicating with known Command and Control system, etc
determining users affected by a security incident (e.g. users whose passwords have been exposed to a keylogger)
suspected access of illegal materials
alleged copyright infringement
violations of University IT regulations
troubleshooting

Note in particular that the prevalence of keylogger infections means that it is increasingly necessary to be able to determine who has used a known compromised system.

This document explains the procedures generally used in order to trace network abuse. It is the responsibility of each college and department to ensure that the necessary information is logged for users of their networks.

Traceability
Tracing single user systems
Communal systems
Multi-user systems
Network Address Translation
DNS Server logs
Timestamps
Incident handling
Retention policy
Potential problems
Further information

1. Traceability

In the vast majority of cases, a problem is seen to involve a particular IP address at a particular time, and will involve an address on a standard college or department backbone connection. Frequently it will be possible for OxCERT to identify the MAC address using the IP address at the time, for instance from ARP data or from DHCP logs.

2. Tracing single-user systems

In many cases, a particular computer is used by just one person. Without NAT, the IP address and timestamp are sufficient to identify both the computer and the person. Where the backbone routers can see the computer's true MAC address, the system can be isolated by a MAC address block. OxCERT will generally use MAC address blocks at the backbone router where possible, as this minimises the danger of collateral damage in environments using dynamic IP address pools.

If OxCERT can only supply an IP address, units may need to be able to map this to a MAC address themselves, for instance from their DHCP server logs. One of IP or MAC address should be sufficient to identify the system and its user.

3. Communal systems

Some abuse comes from communal systems, for instance those in a computer room or library. To determine who was using the system at a particular time, the system should require authentication before use, ideally with logs stored on a separate machine. In the case of "kiosk-mode" terminals where the functionality is severely restricted, it may be appropriate to allow usage without authentication. However if general access is permitted (for instance, access to arbitrary websites) then we would strongly recommend that adequate authentication and logging measures are in place.

Users should be reminded to log out when they are finished and to screen-lock the system should they leave it briefly unattended, in order to avoid others abusing the system under their account.

4. Multi-user systems

The trickiest to handle are multi-user systems, for instance general-purpose servers such as the GNU/Linux service. Such systems permit concurrent usage by multiple users. Utilities such as process accounting allow for some degree of per-user accountability.

Some servers are more specific in purpose, for instance mail relays or web proxies. These may authenticate users, and if not should at the very least log the IP address from which a connection originated.

5. Network Address Translation

NAT (strictly NAT/PAT) setups with multiple systems sharing a single public IP address, make tracing problems far harder, owing to the loss of the usual one-to-one mapping between host and public IP address. It is still essential that network traffic be traceable to a single host on the internal network.

Many NAT setups provide a certain amount of logging, for instance a log of all HTTP connections. While useful, these data are not sufficient, and will fail to log anything for certain types of malware.

In order to reliably trace the source of an outgoing network connection from a system behind NAT, it must be possible to map from the public data (timestamp, protocol, source and destination IP addresses, source and destination ports where appropriate) to the IP addresses and port numbers used on the internal network.

A tool such as Argus running on a mirror port may be useful for recording details of every network flow on the inside of a NAT gateway. For almost all purposes the data gathered will be sufficient. However, argus will not log the NAT gateway's port number mappings as packets traverse the gateway. This may cause problems under certain circumstances. Ideally the gateway itself should log the mappings as used within its own state table. Please bear in mind that every outbound connection attempt which reaches the backbone network needs to be logged. It is not sufficient merely to log successful connections, as such logs will be missing information that is often vital to OxCERT.

OxCERT has given presentations covering NAT, from which slides are available here, in particular our Introduction to NAT Logging will be of use. Additionally, please see the network team's documentation on Network Address Translation for more general details on NAT.

6. DNS server logs

OxCERT may under certain circumstances need to make use of DNS resolver logs when handling incidents. Where clients are configured to use a local DNS resolver as opposed to the central University servers, OxCERT will identify potentially malicious DNS lookups originating from the IP address of the local resolver. To aid investigations, OxCERT strongly recommend that local DNS resolvers are clearly identifiable as such (via hostname or DNS comment), and, subject to local privacy policies allowing it, retain logs of queries so that the source of malicious DNS requests can be identified.

7. Timestamps

It is essential that all logs bear precise, accurate timestamps. OxCERT recommend that logs bear timestamps to a precision of at least the nearest second, and the logging systems' clocks are synchronised with the central Network Time Protocol (NTP) service. Please ensure that you are familiar with the timezone used in your logs (local time? always GMT?), and beware of confusion that may result from transitions to or from daylight savings time.

8. Incident handling

Where centrally-collected logs are insufficient to provide OxCERT the information necessary for an investigation, OxCERT may request a college or department to provide information from their own logs. For example, this may be system logs from a compromised server, or network and port translations from a NAT device.

Depending on the nature of the incident, OxCERT may request that the unit send their full logs for a stated time period. OxCERT will aim to specify as short a time period as is reasonably possible. Nevertheless if malicious traffic is observed on several occasions during a time period (for instance overnight), then OxCERT may request logs for the full time period as there may be more than one infected host active during that time.

Units may be concerned about providing personally-identifiable information (PII) in logs to OxCERT. Data provided to OxCERT will be handled in confidence, accessible only to members of OxCERT and for no longer than is necessary for the purposes of the investigation. Incident summary information will be stored in accordance with OxCERT's data retention policy.

Note that NAT logs will provide OxCERT with information regarding traffic from a unit's internal IP addresses, but in general will have no means of linking those to individuals; further data would be required in order to do so. Consequently most NAT logs should not contain PII; in cases where they do then units should feel free to remove or anonymise PII before sending data to OxCERT.

For further information regarding incident handling, please see the Incident Handling section of the Information Security Toolkit.

9. Retention policy

We recommend that you have a policy in place as to how long logs should be retained. Too long and there may be significant storage overheads and issues with data protection legislation, but too short and logs may expire before an incident can be investigated.

The Data Protection Act 1998 requires that personal data must not be kept for longer than it is needed, a somewhat vague statement! Retention for a reasonable period to prevent/investigate abuse is fine. Industry best current practice suggests that routine logs be kept for 3-6 months; longer if they are still required for a specific investigation. While the government is encouraging providers to keep logs for much longer for anti-terrorism reasons, such moves currently remain voluntary and affect only public networks; the University network and JANET are private. Whilst in most cases it will not be necessary to go back through logs more than a few days OxCERT's recommendation is that you store a minimum of 90 days' data.

10. Potential problems

Where logging is purely in terms of IP and MAC addresses, there are dangers of placing absolute trust in the data gathered. Users assigned one particular IP address can trivially switch to another. This can (usually) easily be spotted through use of tools such as arpwatch. Spoofing MAC addresses is only marginally more difficult, but is harder to track. Logging the MAC addresses associated with each switchport, and any changes, is a possible solution for some setups, for instance if each switchport connects to a different student's room. Ports in public areas are more difficult, and if occurrences of MAC-address spoofing become more frequent, then the long-term solution would be to go for a technology such as 802.1x, requiring the user to authenticate in order to gain network access.

For certain attacks, in which the attacking host is not interested in receiving a response, the source address of a packet may not even be an IP address actually used by the host. Egress filters at the boundary of your network and the University backbone prevent packets with source addresses other than those within your subnet allocation(s) from reaching other networks, but this is no defence against them spoofing another address within your allocation.