7 Networks

Default Network Domain

The Oracle Enterprise Manager Ops Center software always has a Default Network Domain and all public networks are members of it. If you have upgraded your product software, the existing managed networks are now in the Default Network Domain. A new public network becomes a member of the default network domain. If you direct the new network to a user-defined network domain, the network is also a member of that network domain. Like the default network domain, a user-defined network domain provides network resources to a server pool or virtualization host.

User-Defined Network Domains

You create a network domain to support the use of virtualization hosts, server pools, or a virtual datacenter. For example, a virtual datacenter uses server, storage, and network resources in a dynamic way, allocating and releasing resources whenever necessary. The network domain provides the network resources to the virtual datacenter.

When you create a network domain, you can set a limit on the number of networks that can be created in the network domain. Increase the number of networks when accounts in a virtual datacenter are not able to create vnets.

Editing Attributes of a User-Defined Network Domain

You can change the name and description of the network domain and you can change the number of dynamic networks that are in use simultaneously.

To Edit Attributes of a Network Domain

1. Expand Libraries in the Navigation pane.

2. Expand Networks in the Navigation pane.

3. Select the network domain.

4. Click Edit Attributes in the Actions pane.

   The Details tab is displayed in the center pane. The Name Description, and Number of Networks fields are now editable.

5. Edit the name or description or increase the number of networks.

6. Click Save.

Requirements for a Network

A network requires a physical network interface or a link aggregation and the following mandatory specifications:

• IP address of the network

• Netmask

• If you use static IP addressing, the IP address of the management interface

  If you use dynamic IP addressing, the range of allowed IP addresses and the gateway address

Public Networks and Private Networks

Oracle Enterprise Manager Ops Center creates virtual networks and provides them to a virtual datacenter. These networks are created from the resources of the network domain as needed. These networks are displayed in the vDC Management section of the Navigation pane.

A virtual network defines a virtual IP subnet that is divided into public external networks and private external networks. You specify the size of the virtual network, which sets the maximum number of IP addresses for a public external network.

The public external network can be shared among the accounts in a virtual datacenter. You can attach virtual servers from different accounts in a single public external network. When you create a virtual server, you specify the number of public external networks to which it can be associated.

You can reserve the IP address in a public external network for an account use only. When you create a virtual server, you can provide the reserved IP address. Otherwise, the virtual server is allocated the IP address from the server poo.l

Networks for Server Pools

A server pool must have at least one network. When a server pool has more than one network, all virtualization hosts in the server pool are associated with the same set of networks. When you add a virtualization host to a server pool, the virtualization host has access to all the networks defined for the pool and can be an active member of the pool. This ensures that all virtual hosts have network access, even when you migrate a virtual host from one virtualization host to another one within the pool.

The following figure illustrates an example of network connections to two virtualization hosts in a server pool. This server pool has two virtualization hosts and two network associations.

Figure 7-2 Network Connections for a Server Pool

Description of "Figure 7-2 Network Connections for a Server Pool"

Networks for Oracle Solaris Zones

You specify the networks that support a global zone. The network attached to a global zone also supports its non-global zones. You can attach one or more networks to a global zone using the Attach Networks action.

Although the fabric that supports the network does not change, you can specify the way the network works differently for each type of zone. A network that is assigned as shared on a global zone can be assigned as exclusive on another global zone.

• In Shared IP mode, the network interface is used by more than one zone. You define the network interface when you assign the global zone to the network.

• In Exclusive IP mode, the network interface is dedicated to the zone. An exclusive network must be declared for the global zone when you assign network to the global zone. Then you configure the IP configuration for the non-global zone.

For a complete description, see Manage Global Zone Networks in Chapter 14.

Networks for Oracle VM Servers

T he Oracle VM Server Control Domain's network support is specified when the Oracle VM Server software is installed to provide networking facilities for the logical domains. For each network connection, a virtual switch is created.

For a complete description of networks for Oracle VM Server for SPARC, see Attach Networks in Chapter 15.

For a complete description of networks for Oracle VM Server for x86, see Manage Networks in Chapter 16.

Networks for Virtual Datacenters

Each virtual datacenter uses server, storage, and network resources in a dynamic way, allocating and releasing resources when necessary.

The virtual datacenter inherits its network resources from the network domain that supports the server pool. These networks form the public external networks for the virtual datacenter. These networks can then be assigned to the accounts in the virtual datacenter. When the user of an account create a private vNet, either a dynamic private network is created or the static private network is made available for use in that account.

For a complete description of networks for virtual datacenters, see vNets Creation and Network Setup in Chapter 18.

Bandwidth Management

Bandwidth management for data links is available for Oracle Solaris 11 operating system. A data link is a physical NIC, an aggregated link, or a virtual NIC. You define the flow on the data link, prioritizing the network traffic on the link and setting the maximum bandwidth limit.

When a new data link is created, its default bandwidth flow is created by Oracle Solaris 11 operating system. You cannot remove this flow. The flow is removed only when the physical link is removed. However, you can create new flows for the data link. The flow for a network interface is specified by its local and the remote IP address, Internet Protocol, and the type of service.

To create a new flow for a data link, specify a name for the flow and the physical network interface. Then set the new bandwidth properties:

• Priority – Set the priority of the network traffic on the link as high, medium or low.

• Bandwidth Limit – Enable the bandwidth limit to allocate guaranteed bandwidth to the specified link. Enter the maximum value for bandwidth limit in Kbps, Mbps or Gbps

• Set attributes for the data flow to identify its network traffic:

  • Local and Remote IP - The source and destination IP address

  • Transport – The Internet Protocol used such as TCP, UDP. SCTP, ICMP.

  • Ports – The source and destination ports for TCP, UDP, and SCTP.

  • DS Field – The type of service field in the IP packets' header.

To View the Flows for a Data Link

1. Expand Assets in the Navigation pane.

2. Select an Oracle Solaris 11 operating system.

3. Click the Networks tab in the center pane.

4. Click the Bandwidth Management subtab in the center pane.

To modify a flow, click the Modify icon. To create a new link, click the Add icon.

IP Multipathing Groups

Using IP Multipathing (IPMP), two or more physical network interface cards (NIC) form a group that use one IP address. If one NIC fails, the other NIC in the group maintains network access.

When you create a network, you define its network interface. The interface can be a physical network interface card (NIC) or, in the case of an Oracle Solaris OS asset, it can be an IPMP group or link aggregation You can implement both methods on the same network because they work at different layers of the network stack. See the Oracle Solaris 11 documentation at http://docs.oracle.com/cd/E23824_01/index.html and Oracle Solaris 10 documentation at http://www.oracle.com/technetwork/documentation/solaris-10-192992.html for information about these services and how to implement IPMP.

IPMP provides increased reliability, availability, and network performance for systems with multiple physical interfaces. It provides physical interface failure detection and transparent network access failover.

Occasionally, a physical interface or the networking hardware attached to that interface might fail or require maintenance. By using IPMP, you can configure one or more physical interfaces into an IPMP group. After configuring the IPMP group, the system monitors the interfaces in the IPMP group for failure. If an interface in the group fails or is removed for maintenance, IPMP migrates, or fails over, the failed interface's IP addresses. The failover feature of IPMP preserves connectivity and prevents disruption of any existing connections.

The association between an IPMP group and a network must be unique; an IPMP group can be associated with only one network and a network can be associated with only one IPMP group or individual NICs.

In an IPMP group, you define whether each interface is a failover or a standby one. The actions of each type differ if the current network interface fails:

• Network access changes from the failed interface to the failover interface in the IPMP group and uses the failover interface data address. You must provide the data address for an interface that is defined as failover.

• Network access changes from the failed interface to the standby interface in the IPMP group but does not change its data address. The data address of the failed interface migrates to the standby interface.

Link-based failure detection in an IPMP group is always enabled if your interface supports this type of failure detection. You can also set for probe-based failure detection by providing a test address for each interface in the group.

You can create a single IPMP group while provisioning an operating system. If you create IPMP groups manually, Oracle Enterprise Manager Ops Center identifies and displays the groups on the UI.

Link Aggregation or Bond

In an aggregated link, two or more NICs form a group and all members of the link aggregation provide network access at the same time. In addition to the high availability and load balancing that an IPMP group provides, an aggregated link can also provide increased throughput if the network ports are also aggregated.

When you create a network, you define its network interface. The interface can be a physical network interface card (NIC) or, in the case of an Oracle Solaris OS asset, it can be an IPMP group or link aggregation You can implement both methods on the same network because they work at different layers of the network stack.See the Oracle Solaris 11 documentation at http://docs.oracle.com/cd/E23824_01/index.html and Oracle Solaris 10 documentation at http://www.oracle.com/technetwork/documentation/solaris-10-192992.html for information about these services and how to implement link aggregation.

Link aggregation is a standard defined in IEEE802.3ad. An aggregated link consists of several interfaces on a system configured as a single, logical unit. Link aggregation increases the speed and high availability of a connection between a server and a switch. The most common protocol used to manage link aggregation is LACP (Linked Aggregation Control Protocol).

You can create one link aggregation while provisioning an OS. You can configure the interfaces together as a logical unit and define the link aggregation information if the following conditions are met:

• All the members of the aggregated link are connected to the same switch.

• The members of the aggregated link are of the same type. For example, NICs with the e1000g interface cannot be mixed with NICs that use the bge interface.

• For Oracle Solaris OS, the required driver is GLDv3.

When interfaces have been aggregated, they are treated as a single network interface. Oracle Enterprise Manager Ops Center displays the link aggregation in the list of available NICs as if it were an individual interface. When you assign a network with a link aggregation to an Oracle VM Server, logical domain, global zone, or a non-global zone, select the link aggregation from the NIC list. You can view the link aggregation details on the Oracle VM Server's or global zone's Network tab.

IPv4 and IPv6 Protocols

Some environments have a mix of IPv4 and IPv6. Oracle Enterprise Manager Ops Center is "IPv6-aware." If an asset has an IPv6 network interface, Oracle Enterprise Manager Ops Center can read it and displays its information, but it cannot provision an IPv6 network or use IPv6 networks to discover, monitor, or provision assets.

Routing Mode

A virtual host uses the network assigned to it according to the host's routing mode. You specify a virtual host's routing mode during its initial configuration if you do not accept the default mode, Automatic Routing. Oracle Enterprise Manager Ops Center supports the following routing modes:

• Automatic Routing – This is the default routing mode. Applying the static routes depends on the following conditions:

  • If your site defined a default gateway or static route or retrieved one from the DHCP server, this route is used and dynamic routing is disabled.

  • If no default gateway or static route is available, dynamic routing is enabled.

• Dynamic Routing Off – The virtual host uses the default gateway and any static routes configured for the network. The default gateway is retrieved from the DHCP server.

• Dynamic Routing On – The virtual host uses routes provided by the dynamic routing service. The default gateway and any static routes configured for the network are ignored.

Static Route for the Network

Static routes specify the route for external access. Although you define a default gateway for a network, it might not reach a particular subnet. In this case, you must also provide a static route for the subnet.

When you create a network, you can specify the static route. To add static routes after the network has been created, use the following procedure.

To Add a Static Route for the Network

1. Click Managed Networks in the Navigation pane.

2. Select a network from the list of networks.

3. Click Edit Network Attributes in the Actions pane.

4. Click the Add icon in the Static Routes table. A row is added to the table.

5. Type the values for destination IP, netmask, and gateway.

6. Click Finish.

You can delete a static route and change the order of the routes using the icons in the Static Routes table.

Address Allocation Method

When you create a network, you specify how its IP address is assigned:

• Static IP: A specific IP address is assigned to the network

• Automatic: An available IP address is assigned to the network.

Maximum Transmission Unit (MTU)

The default size for the network's Maximum Transmission Unit (MTU) is 1500 bytes. If your network interface card is one of the following types, you can change the size of the MTU to a size between 576 and 9216 bytes. However, to assign the network to a logical domain, the minimum MTU size is 1500 bytes.

• nxge

• ixgbe

• hxge

• e1000g

• ce

• bge

• ipge

When you specify a size greater than 1500 bytes, Oracle Enterprise Manager Ops Center modifies the network interface card's MTU size. For other types of network interface cards, the MTU is changed when the card's driver firmware is updated to support the new MTU size. However, to change the MTU value for an IPMP group, you must edit the MTU value manually.

VLAN and VLAN Tags

For fabrics based on Ethernet protocol, the ability to use VLAN tags is an attribute of each network. Use the Edit Network Attributes action to add or change the VLAN capability.

Network Interface Card (NIC)

The Network Interface Card (NIC) is the physical connection to a network. To see the NICs for a server, select the server and then click the Connectivity tab. The Server Processor Connectivity table lists all of the NICs.

Network Switches

Oracle Enterprise Manager Ops Center can manage Sun Ethernet 10GbE Fabric switches and Sun Datacenter InfiniBand switches. These switches reside in the system or blade system and provide the switch fabric.

For the Ethernet switches, both tagged and untagged VLANs are supported.

If you use an InfiniBand switch in an Ethernet network, the ports on the switch have Ethernet names.

For more information about these switches, see the product documentation:

• For the Sun Ethernet 10GbE Fabric switch, see http://docs.oracle.com/cd/E19934-01/index.html

• For Sun Datacenter InfiniBand switch, see http://docs.oracle.com/cd/E19654-01/index.html

Network Switch Configuration

Use these guidelines to configure a network switch for a system running the Oracle Enterprise Manager Ops Center software.

• Use an Virtual LAN (VLAN)-capable switch.

• Discover and manage the switch.

• Create a separate VLAN for Oracle Enterprise Manager Ops Center management and provisioning networks.

For Ethernet connectivity:

• The management network must be a 10/100 connection.

• The provisioning and data networks must be at least a 10/100/1000 (1 GB) connection.

Separate Management, Provisioning, and Data Networks

• Separate networks provide the highest security and the lowest number of points of failure.

• Additional NICs are needed to support this configuration.

Figure 7-3 Separate Management, Provisioning, and Data Networks

Description of "Figure 7-3 Separate Management, Provisioning, and Data Networks"

A configuration with separate management, provisioning, and data networks has these requirements:

• Enterprise Controller/Proxy Controller

  • ETH0 connects the Enterprise Controller/Proxy Controller to the corporate network for external access. Configure the ETH0 IP address, netmask, and gateway to meet corporate connectivity requirements.

  • ETH1 connects the Enterprise Controller/Proxy Controller to the provisioning network and must be on the same network as the ETH0 connections of the Agent Controllers. Only the Enterprise Controller/Proxy Controller and the Agent Controllers must reside on the provisioning network. ETH1 must be a 1 Gb NIC interface.

  • ETH2 connects the Enterprise Controller/Proxy Controller to the management network and must be on the same network as the management port connections of the Agent Controllers. Configure the ETH2 IP address, netmask, and gateway to enable connectivity to the Agent Controllers' management port IP addresses. ETH2 must be a 100 Mb NIC interface.

  • The DHCP service allocates IP addresses to the Agent Controllers for provisioning operating systems.

• Agent Controllers

  • Each Agent Controller's management port connects the Agent Controller to the management network and must be on the same network as the ETH2 connection of the Enterprise Controller/Proxy Controller. The management port must be a 100 Mb connection.

  • ETH0 connects the Agent Controller to the provisioning network and must be on the same network as the ETH1 connection of the Enterprise Controller/Proxy Controller. ETH0 must be a 1 GB connection.

  • ETH1 connects the Agent Controller to the data network through the switch to provide corporate network access to the Agent Controller. ETH1 must be a 1 GB connection.

Combined Management and Provisioning Network and a Separate Data Network

• Reduced system and network security.

• No additional NIC is needed on the Enterprise Controller or Proxy Controller.

Figure 7-4 Combined Management and Provisioning Network and a Separate Data Network

Description of "Figure 7-4 Combined Management and Provisioning Network and a Separate Data Network"

• Enterprise Controller/Proxy Controller

  • ETH0 connects the Enterprise Controller/Proxy Controller to the corporate network to provide external access. Configure the ETH0 IP address, netmask, and gateway to meet corporate connectivity requirements.

  • ETH1 connects the Enterprise Controller/Proxy Controller to the management and provisioning network and must be on the same network as the MGMT and ETH0 connections of the Agent Controllers. Only the Enterprise Controller/Proxy Controller and the Agent Controllers must reside on the management and provisioning network. The ETH1 IP address, netmask, and gateway must be configured to enable connectivity to the Agent Controller's management port IP addresses. ETH1 must be a 1 Gb NIC interface.

  • The DHCP service allocates IP addresses to the Agent Controllers for provisioning operating systems.

• Agent Controllers

  • Each Agent Controller's management port connects the Agent Controller to the management and provisioning network and must be on the same network as the ETH1 connection of the Enterprise Controller/Proxy Controller. The management port must be a 100 Mb connection.

  • ETH0 connects the Agent Controller to the management and provisioning network and must be on the same network as the ETH1 connection of the Enterprise Controller/Proxy Controller. ETH0 must be a 1 GB connection.

  • ETH1 connects the Agent Controller to the data network through the switch to provide corporate network access to the Agent Controller. ETH1 must be a 1-GB connection.

Combined Provisioning and Data Network and a Separate Management Network

Figure 7-5 Combined Provisioning and Data Network and a Separate Management Network

Description of "Figure 7-5 Combined Provisioning and Data Network and a Separate Management Network"

• Enterprise Controller/Proxy Controller

  • ETH0 connects the Enterprise Controller/Proxy Controller to the corporate network to provide external access. Configure the ETH0 IP address, netmask, and gateway to meet corporate connectivity requirements.

  • ETH1 connects the Enterprise Controller/Proxy Controller to the provisioning and data network and must be on the same network as the ETH0 connections of the Agent Controllers. Only the Enterprise Controller/Proxy Controller and the Agent Controllers must reside on the data and provisioning network. ETH1 must be a 1 Gb NIC interface.

  • ETH2 connects the Enterprise Controller/Proxy Controller to the management network and must be on the same network as the management port connections of the Agent Controllers. Configure the ETH2 IP address, netmask, and gateway to enable connectivity to the Agent Controller's management port IP addresses. ETH2 must be a 100 Mb NIC interface.

  • The DHCP service allocates IP addresses to the Agent Controllers for provisioning operating systems.

• Agent Controllers

  • The management port connects the Agent Controller to the management network and must be on the same network as the ETH2 connection of the Enterprise Controller/Proxy Controller. The management port must be a 100 Mb connection.

  • ETH0 connects the Agent Controller to the data and provisioning network to provide corporate network access to the Agent Controller. ETH0 connection must be on the same network as the ETH1 connection of the Enterprise Controller/Proxy Controller. ETH0 must be a 1 GB connection.

Combined Provisioning, Data, and Management Network

• Least secure system and network

• No additional NIC is needed for the Enterprise Controller/Proxy Controller.

Figure 7-6 Combined Provisioning, Data, and Management Network

Description of "Figure 7-6 Combined Provisioning, Data, and Management Network"

• Enterprise Controller/Proxy Controller

  • ETH0 connects the Enterprise Controller/Proxy Controller to the corporate network to provide external access. Configure the ETH0 IP address, netmask, and gateway to meet corporate connectivity requirements.

  • ETH1 connects the Enterprise Controller/Proxy Controller to the combined management, provisioning, and data network and must be on the same network as the MGMT and ETH0 connections of the Agent Controllers. Only the Enterprise Controller/Proxy Controller and the Agent Controllers must reside on the combined network. ETH1 must be a 1 GB NIC interface.

  • The DHCP service allocates IP addresses to the Agent Controllers for provisioning operating systems.

• Agent Controllers

  • Each Agent Controller's management port connects the Agent Controller to the management, provisioning, and data network and must be on the same network as the ETH1 connection of the Enterprise Controller/Proxy Controller. The management port must be a 100 MB connection.

  • ETH0 connects the Agent Controller to the management, provisioning, and data network, and must be on the same network as the ETH1 connection of the Enterprise Controller/Proxy Controller. ETH0 also connects the Agent Controller to the data network through the switch to provide external corporate network access to the Agent Controller. ETH0 must be a 1 GB connection.

Firewall Rules

The Enterprise Controller must reach some external sites. If you have explicit firewall rules enabled to allow access to these services from your Enterprise Controller, you must update these rules to allow access to the following addresses:

Network Port Requirements and Protocols

This section lists the ports and protocols used by Oracle Enterprise Manager Ops Center.

The Enterprise Controller's default port is 443. If port 443 is in use, the Enterprise Controller uses Port 11165. The following table describes the required ports and their protocols.

^Footnote 1 The FTP protocol is used only by servers with the ALOM service processor or by SPARC Enterprise M-Series servers.

^Footnote 2 The telnet protocol is used only by servers with the ALOM service processor.