                          Installation Notes

                QLogic's Linux iSCSI Offload Driver

                             Cavium Inc.
                         2315 N.First Street,
                         San Jose, CA 95131

                Copyright (c) 2015-2018 Cavium Inc.
                        All rights reserved


Table of Contents
=================

  Introduction
  Distros Supported
  Differences from bnx2i
  Core Module
  QEDI Dependencies
  QEDI Module Parameters
  Build, Install, Load and Unload
  Module Install
  Additional Notes
  Known Issues

Introduction
============

This file describes the qedi Linux offload driver.

Distros Supported
=================

RedHat Enterprise Linux 6.5 and above
RedHat Enterprise Linux 7.0 and above
SLES 11 SP4, SLES 12 SP1 and above
Citrix Xenserver 6.5
Ubuntu 14.04, 16.04
CentOS 7.2

Differences from bnx2i
=======================

There are some signficant differences between qedi and the previous generation
QLogic iSCSI offload driver, bnx2i:

1. qedi directly binds to a PCI function exposed by the adapter.
2. qedi does not sit on top of the net_device.
3. qedi is not dependent on a network driver like bnx2x.
4. qedi is not dependent on cnic, but it has dependency on qed.

Core Module
===========

qedi require eCore core module i.e. qed, be loaded.

QEDI Dependencies
=================

qedi is dependent on the following kernel modules:

- scsi_transport_iscsi
- libiscsi
- iscsi_ibft
- uio
- qed

For userspace, need to use qedi supported open-iscsi initiator tools.
Note: On newer distros (which supports systemd environment) like
      RHEL7.0/7.1 and SLES12.0. Using open-iscsi initiator tools
      requires following packages to be installed, with compatible
      versions on corresponding OS:

	1. libuuid-devel
	2. libblkid-devel
	3. libmount-devel
 
Please note that it is upstream open-iscsi requirement that depends on
above mentioned packages and not iscsiuio.

QEDI Module Parameters
======================

debug		- Addition kernel logging for trouble shooting exception conditions.
int_mode	- Force interrupt mode other than MSI-X (default is 0, MSI-X).
fw_debug	- Firmware debug level 0 (default) to 3
ooo_enable	- Enable iSCSI out of order:0 (Disable); 1 (Enable) Default
io_tracing	- Enable logging of SCSI requests/completions: 0 (Disable, Default), 1 (Enable).

Build, Install and Load
=======================

To build and install the qedi module, issue the below commands from the top level 
directory of the fastlinq package:

     # cd fastlinq-8.10.2.0
     # make
     # make install

To load the qedi module, first load the qed.ko module:

     # modprobe -v qed

Then load qedi.ko itself:

     # modprobe -v qedi

To unload the qedi module, first logout all the target sessions and
stop the iscsiuio service, then issue below command,

     # modprobe -rv qedi

Module Install
==============

The Makefile 'install' target needs to run so that the OS tools can correctly
load and unload the qedi module. Also this needs to be done to build the qedi
module into initrd/initramfs so it can be loaded at boot.

Since the qedi is not currently inbox in all distros, a kernel cmdline needs
to be added to the grub configuration file so the driver loads at boot:

o RHEL 6 - rdloaddriver=qedi
o RHEL 7/SLES 12 - rd.driver.pre=qedi

Additional Notes
================

1. Refer Open-iSCSI Readme for Advanced Target configurations

   https://github.com/open-iscsi/open-iscsi/blob/master/README

2. To view the logged in target details, use below iscsiadm command

     # iscsiadm -m session -P3

3. For Target connection and session information, check the sysfs interface,

     cat /sys/class/iscsi_connection/
     cat /sys/class/iscsi_session/

4. To enable SCSI multiq on upstream kernels,

    echo Y > /sys/module/scsi_mod/parameters/use_blk_mq

Known Issues
============

- Only MSI-X interrupt mode is supported.
- CQ85624 - On RHEL 7.2, session recovery tmo value changes from 120s to 5s.
  For this update the multipath.conf defaults section to add
  "fast_io_fail_tmo off". This will prevent the change of sess recovery tmo
  value.
- SuSE BZ#1025177 - On SLES 12 SP2, observe intermittent system crash due to
  kernel BUG at mm/slab.c
- SuSE BZ#1025438 - On SLES 12 SP2, L4 iSCSI boot OS fails to come up after
  L2 iSCSI to L4 iSCSI migration
- RedHat BZ#1425773 - On RHEL 7.3, L4 iSCSI boot OS fails to come up after
  L2 iSCSI to L4 iSCSI migration
- Redhat BZ#1372411 - On RHEL 6.8, NetworkManager fails to find ethX when
  configuring parameters from iBFT over software iSCSI
- In order to perform L4 iSCSI boot, user needs to perform L2 iSCSI boot,
  update the open-iscsi components and then migrate to L4 iSCSI boot.
  Please refer "iSCSI BFS – Migrating from non-offload (iBFT) to offload (L4)"
  document.
- iSCSI offload is not officially supported in Xenserver 7.0 (QL-519), though
  an out of box solution works.
- CQ91138 - After installation of open-iscsi rpm, auto load of iscsid service
  fails. Disable SELINUX to resolve this behavior.
- CQ93108 - Auto start of iscsiuio service fails on SLES 11 SP4.
  Add qedi module under /usr/sbin/rcopen-iscsi start routine, which
  is looked before starting iscsiuio service.
- CQ98259 - RHEL7.4 - L4 DHCP IPv6 target login fails during Boot From SAN
  installation. Issue is in inbox iscsiuio. Update to the latest available
  iscsiuio.
