Design Summit December 07

Thursday

Alan Maguire slides

1. clearview UV/brussels/NWAM all store config data - crossbow will, too;
   need to figure out how it fits
2. Store data in smf repository, organized by ncu
   1. svc:/network/datalink:<name>
   2. svc:/network/ip:<name>
3. Using the vanity name as the service name means that name changes are harder; but can't really
   get rid of <name> interdependency, as the whole idea behind vanity names is to make one single
   namespace for administration
4. So, renaming is hard: a link name shows up in many places, including inter-link dependencies,
   and possibly as attributes of another link (e.g. links net0 and net1 together form aggr0)
5. assertion: an aggregation doesn't know the identity of its underlying links. This doesn't make
   sense to us, need to figure out what was meant here. Maybe this statement was really about
   dependencies: an aggr does not depend on every one of its underlying links being up. Does aggr
   link up/down state toggle on up/down state of last link?
6. start methods - should they do setup or should they switch between nwam doing setup when it is
   enabled and start method/script doing it in non-nwam mode? How should that work be partitioned?
   Would like to have as much in common between nwam and non-nwam case as possible.
7. Creating configuration objects: how are the smf instances actually created? Who
   does it? High level answer: datalink-management and ip-management services' start
   methods. Okay; but how? Right now, nwamcfg is the only tool we have to create
   these objects. Seems odd to use nwamcfg in a non-policy-engine-specific context.
   Could move instance creation into policy engines (which already do enabling/disabling
   of instances); this means there's not much reason to have ip-mgmt service. Eliminate
   it? Problem with this is we lose parallel structure for L2/L3; doesn't really buy
   us much, either. Value of consistency seems to outweigh any reduction in complexity.
   1. alan - instead of ipcfg service have individual datalink methods
   2. nwam has no idea about hostname files 'cept for upgrade
   3. nwam auto mode is just plumb ip on all links
   4. alan - still need tool to create instances
   5. renee - real solution is to create 'ipadm', L3 analog to dladm; this is a challenge
      for phase 1 because it's more code, more testing.
   6. but we could use nwamcfg in scripts and replace later
8. conflict between putting temporary state (ifconfig) in SMF as clearview wants and not as we have
   talked about in NWAM design (ifconfig changes are viewed as "hands off" by nwam; analogous to
   dhcpagent's behavior when you modify an interface it is controlling). This is question A1.
   1. config vs state - should only store configuration in repository, so no dhcp addrs, link locals,
      etc.
   2. but still need to understand (uv, brussels) need to store temporary state
   3. crossbow QoS config; is it permanent or temporary?
   4. use case for temporary is to play with config, get it set up exactly the way you want,
      and then make permanent config out of it (push the "save this setup" button!)
   5. Max and Alan have looked at using temporary properties in smf
   6. Problem here is that a "single" configuration change (e.g. unplumb ip from 3 links and make
      an aggregation of those links, then plumb ip on the aggr) may affect more than one property;
      how is the set of changes managed?
   7. enhanced profiles might help here? But we can't count on that in the phase 1 timeframe...
9. Instance start methods need to differentiate based on policy engine (e.g. in case of ip
   instances, data comes from repository when nwam is enabled, and from hostname files when
   it's not). Alternatively, can instance method just hand control back to policy engine when
   it's done? That is, instance method just does plumb and let network/physical:default do
   the right thing. The problem there is it makes interaction complex and needs quite a bit
   of change to the existing network/physical:default start method. Perhaps it's not so ugly
   to have instance start method decide which path to follow. Path could be pretty empty if
   the policy engine does the work; more complex if not.
   so state of instance tells about config
   Will go into this in more detail in question E1.

10:00 <break>

Renee's questions

A1 Temporary state

• 3 things we can represent
  • configuration
  • user made non persistent change ('ifconfig unplumb bge0')
  • couldn't actually do it (link should be enabled, but cable not plugged in)
• Which of these should be in the repository? NWAM team suggests that only the first, the stored
  configuration, should be in the repository. That's what will be applied when the service is
  [re]started.
• Clearview UV does store some temporary (non-persistent) config data in /var/run/dladm.conf; what
  is this and should it be in the repository as well?
• We also need to be able to communicate to user differences between stored config and actual config;
  current gui doesn't have an obvious place to plug in this sort of info; this might be something
  that needs to wait until post-phase 1?

  Decision Only linkid-link name mappings are stored in /var/run/dladm.conf (refer to clearview-discuss thread); we can continue to use that file for that data, rather than moving it into the repository.

  Question The question does go beyond the data stored in /var/run/dladm.conf, though; the UV team believes temporary links should also be represented in the repository. Discussion on this topic is ongoing; refer to the opensolaris threads here and here. There seem to be two issues: one architectural, and one design-related. Architectural: should temporary instances appear in the repository? Some argue that they must be in order to fully represent the network configuration; it would be confusing to admins to do otherwise. Others argue that this breaks the smf model; the smf repository represents the things that are being managed by smf, not everything that exists in the kernel. Design: if temporary instances are represented in the repository, how is that done? The current smf framework has a notion of temporary properties, but not temporary instances. We would need a mechanism to make sure that instances representing temporary config are destroyed when appropriate (on reboot, restart of the policy engine, etc.). It might be that some future smf enhancements will better support this.

• If we do need to store temporary config in the repository, one way would be via enhanced SMF
  profiles; but we don't know schedule; probably won't be available in phase 1 timeframe. Maybe
  we could fake up a solution using "overlay"

A2 canned profile definitions

• We need three basic ones:
  • no-net location
  • automatic NCP
  • automatic location
    • no-net location is a degenerate form of the automatic location? Seems right.
• automatic NCP: one interface only (follow phase 0 policy)? Or maybe all available interfaces?
  Automatically bringing up all available is okay for wired, as plugging in a cable expresses intent.
  But we don't have such an indicator for wireless.

Decision Automatic NCP will implement the phase 0 policy: one interface active at a time, and prefer wired over wireless.

• Implementation goal: even though we only intend to bring up one interface, we want to attempt to
  gather information about multiple interfaces simultaneously (i.e. do dhcp requests in parallel).
  alan: should the parallelism be represented in SMF repository?

AI Need to define automatic and no-net locations; this depends on the definition of locations in general. See question A8.

A3 start methods; specifically for wireless links

• network/physical uses wificonfig startconf (which either uses a stored wificonfig profile for
  connect parameters, or does an autoconf) when bringing up a wireless link/interface; do we need
  to maintain this behavior when using the default (non-nwam) policy engine? If so, can we emulate
  this behavior with other tools (i.e. not use wificonfig)? Original plan was to remove wificonfig
  when nwam integrated.

  Decision Remove wificonfig, but continue to have comparable behavior. Use a common (shared with nwam) visited list, that's stored in the repository (as a property of the datalink service).

  Questions How we define "comparable" is the rub... List is stored in SMF repository Will not fall back to autoconf in the absence of recognized wlan (as wificonfig currently does) Entries will be added at time of connection (how? add ability to manipulate the list to dladm?) Problem with shared list: nwam's visited list semantics are different from wificonfig's profiles, which identify "preferred" wlans rather than "visited" wlans. nwam does allow the list to be ordered, so it does include a "preferred" notion as well. What about keying material? Should it also be common? If so, will probably need to change naming convention ("nwam-<essid>-<bssid>") for secure objects. Behavior needs to be pulled into the start method for individual datalinks; but this is a problem because... Currently, IP must be plumbed before a wireless driver will connect to an AP, because the connected driver must have at least one reference to prevent the possibility of it being unloaded at any time. This means the network/ip service instances would need to be enabled before wifi connections can be made, so putting the wlan connection logic in the start method of the datalink service instances is problematic. Could we solve this by considering the external connection to an AP a reference, and bump ref count upon connection (meaning that the IP stream would not be required to insure the driver is not unloaded)?

• What about using write-only properties to store keying material in SMF?

  Decision Not in phase 1; and we'll need to feel more comfortable that that's an appropriately secure mechanism before using it.

• What exactly should the datalink and ip start methods look like? Per-interface behavior
  in /lib/svc/method/net-physical should break out into appropriate start methods; what about
  things like 'dladm up-aggr' and the newly added hypervisor bits?

Friday

A7 how exactly does one ip ncu per link work?

1. Need to be able to represent multiple address acquisition methods
2. We could represent as a bitmask, booleans, or a multivalued list.
   1. Using a bitmap would not allow multiple types like RA + DHCPV6.
   2. We could have booleans for each.
   3. We could use multivalued properties for each. Properties are always multivalue lists (no
      ordering) so extending to this is easy.
      1. SMF supports v4, v6 address types.
      2. We could use multivalue for v6 DUID or DHCP token if necessary.
      3. Other parameters are generaly good for everything but private addresses and their defaults
         still work,

         Decision Addresses/Methods are single multivalued parameters.

         rldQ Need to specify properties. v4addrsrc, v6addrsrc, v4static, v6static? maybe just one addrsrc, with v4 and v6 merged?

A6 rename

1. Alan mentioned that liane or Michael mentioned having transaction functionality
2. UV requirement: interfaces must be quiescent
   1. renee had conversation with meem from which she has an unmemory of a good requirement.
   2. A simplifying assumption is that names will be stable over a long period.

      mphQ Is that really true even with GUI allowing it front and center. Or do we following this with a requirement that it be a heavyweight operation. rld comment: I think the gui needs to make it clear that this is a heavyweight operation; that it will result in the link being taken down temporarily.

Discussion of Max's notes from reading our Thursday meeting minutes

1. Not sure if comments on not wanting to save dependencies makes sense
   1. use include any, but it sounds like cfgadm is more like include all?
   2. poliferation of linkid really wanted even outside of admin interface?
   3. alan - do we need to wait for all links in aggregations to come up?
   4. linkid to/from vanity name is owned by dladmgt right?
   5. dr and/or pcmcia have their own profile mechanism?
   6. assertion: max saying with cfgadm/dr more can be done then with smf
   7. agree with max that doing setup/teardown in service methods
      1. mostly already done so for many links start method is true
         1. so really its state and presence of instances that is interesting
   8. mws - up-aggr should really do -t since it is the result of reading permanent config
      1. why? seems twisted to have start method with create-aggr -t?
      2. renee - maybe no so twisted. create-aggr two otuputs:
         1. without -t permanent config and plumbing
         2. with -t just config
         3. does -t need to create linkid that already exists
            1. is this solved by restart == start + stop
2. questions/comments:
   1. do we use smf dependencies to bring up underlieing links?
      1. does the same thing happen with ipmp?
      2. alan: we still need to use dependencies between link and ip
      3. do we only not want to use them between links?
      4. what is the behavior we are really trying to model
   2. renaming
      1. we are happy with quiescing the system
      2. we still need a way to do the transaction atomically
         1. need to talk with mws/liane
   3. need to talk about use of -t with clearview team
3. max didn't see issue with transactions as issue
4. Alan talk with clearview team and only temporary data is really linkid/name mapping for recovery
   1. does that confuse admin view?
   2. sounds like we are back to the philosophical question of what does SMF data represent: config or running?
      1. clearview team seems to be going for running?
      2. SMF philosphy is that it is configuration
      3. is there a difference between service and instance
      4. is this an issue because we are doing something different?
5. max's last item is that ifconfig should create instance - same issue/question

A8 Definition of location: what are the "hardwired" properties?

mph AI Come up with specific list of properties for locations.

B1 repository layout/libraries - also discussed in alan's slides

1. notification - via SMF method broadcast or SMF API
   1. event register, event generation
      1. is that easier in nwamd speicific situation to use nwamd; but nwamd not always there
      2. could use mutliple doors to send out events
2. might be useful to have property group stating what policy engine supports

B2 update API to reflect choice of location over environment?

Decision Yes, we need to s/env/loc/ in library function names.

10:15 Break

F1 upgrade/install

1. Solution for now is /var/svc/... script, which is executed by manifest-import service on boot.
2. jumpstart - nwamcfg with alternative root, etc.
   1. maybe enhanced profiles, although does that only enable services?

C1 UI privilege issues

1. looks like two requirements from Gary is sufficient
   1. RFE 6219893 make dladm read APIs not need privs
      1. Need to look at PSARC 2006/623
   2. how to allow user to change config via workstation owner

D1 alan's specification of state machine http://opensolaris.org/os/project/nwam/p1spec/daemon

1. LINK_STATE shouldn't be routing socket, but should be dlpi notification - need to check if its been
   integrated yet - yes PSARC 2006/514 DLPI Notification API inegrated in 75
2. NO_MAGIC - too generic
   1. maybe need state machines per interface type and then overall intelligence
      1. AI - mph needs to write up multiple state machine idea, specify NO_MAGIC thing better
   2. Alan suggests that the way that the GUI communicates the information back to nwamd is via properties
   3. dladm create-secobj - how will this be done via the workstation role

D2 keeping SMF state in sync with link state

1. can't really do this anymore because not delagated restarter

D3 delegated restarter for locations?

1. really interesting when we have enhanced profiles
2. alan: really ease of use and getting info out of nwam
   1. not an issue now but maybe in future

D4 cold start behavior

1. pasted information out of document

D5 advanced link/interface types

1. example net0, net1 with ip, user takes both off and creates aggregation (dladm create-aggr which creates
   config)
2. would it be better to just support them in the daemon and let GUI support comes as it will
   • warm vs cold start discussion along the above lines.

Friday Afternoon: Crossbow config with Mike Lim

• Crossbow config uses vnic.conf, linkprop.conf, flowadm.conf, flowprop.conf
• vnics are a new link type
• flows would be a new service: network/flow:foo
• Do we need to pull flows into SMF at the same time we do everything else?
• How do we manage timing/coordination?