Data Interface Requirements

Document Purpose

This document defines the complete set of data input requirements for NETSTOCK’s ongoing SaaS solution.
The exact requirements will differ from system to system (depending on what data is available) and from customer to customer (depending on the business structure). This should be finalised in conjunction with NETSTOCK during the initial requirements phase.

Data Extraction Timing

NETSTOCK operates on a daily refresh of data. The most appropriate timing for data extraction out of the host system is usually early in the morning. Files are exchanged with NETSTOCK using secure FTP which is facilitated by a program installed on a local server with internet access.

Lead Times

Lead times are crucial to NETSTOCK but in some cases it may prove difficult to source meaningful data, therefore there are several alternatives available (listed from most to least preferred):

    Provide lead times in the Stock file (if the data is relevant and available)
    Provide lead times in the Supplier file
    NETSTOCK can derive lead times by analysing the receipts of completed purchase orders in the Completed Purchase Orders file (assuming the data is provided)
    Manual configuration of lead times by supplier

Unit of Measure

All stock related quantities in the data provided are assumed to be in a common unit of measure (referred to throughout this document as the “NETSTOCK unit of measure”). This should typically be the stocking unit of measure, however it must be sufficiently granular to eliminate the need for decimal places in NETSTOCK. Any necessary conversions should be undertaken during the data extraction.

File Format

NETSTOCK requires data files in CSV format (i.e. comma separated values).
Here are some important guidelines:

  • File names must be precisely as specified (noting the all-lowercase lettering)
  • Field headings are to be omitted (except for custom ut_### files)
  • For fields that are not being provided, blank values must still be included to ensure conformity to the file specification (each field must be in its correct position). Blank values can be supplied by using an immediate comma to move to the next field.
  • All designated character fields should be wrapped in double quotes (for example, the value ABC123 should be supplied as “ABC123”).
  • If a double quote features within the data itself, replace it with two consecutive double quotes to distinguish it from the start and end double quote delimiters.

Data Types

Several basic data types are referred to in this specification.
The following table provides information on field sizes and formats:

Data type Comment
Char Variable length strings up to 255 characters long
Integer Numeric values without a decimal component
Decimal Numeric values with a 12 digit maximum including 4 decimal places. Decimal places must be delimited by a full-stop character
Date Date values should be formatted as YYYY/MM/DD E.g. 22 August 2011 should be represented as 2011/08/22
Bit Bit fields allow only the binary values of 0 (false) and 1 (true)

List of Files

The following table summarises the input files that drive NETSTOCK software.

Data File Name Importance Description
Locations location.csv Required Physical stocking locations (warehouses and branches)
Suppliers supplier.csv Required Active vendors
Product Master master.csv Required Basic product information
Stock by Location stock.csv Required The current stock position by location
Custom Groups group.csv Required Codes and descriptions for custom data
Sales & Issues sales.csv Required Sales and issues summarised by month (or the equivalent transactional data)
Outstanding Purchase Orders po.csv Required Outstanding purchase / production order lines
Outstanding Customer Orders co.csv If Applicable Outstanding customer order lines
Outstanding Transfers transfer.csv If Applicable Outstanding transfer lines
Completed Purchase Orders pohist.csv Preferred Completed purchase order lines and receipts
Meta Data ('Trigger' File) control.csv Required Information about the interface and data extraction
Bills of Materials bom.csv If Applicable The relationship between finished goods and raw materials
Supersessions supersessions.csv If Applicable Product linkages where a product(s) has been replaced by a new product
Custom Data ut_###.csv If Applicable Custom file(s) for customer specific business rules

ERP Connector

Connecting your ERP system to the NETSTOCK cloud servers involves:

  • The ERP server, on your network, runs your ERP system

  • The Comms server, in the cloud, co-ordinates data flow between the ERP server and the App server

  • The App server, in the cloud, runs the NETSTOCK application

Upon installation

When you sign up:

  • Your instance of the NETSTOCK App is created on one of our App servers

  • The Comms server is configured to allow secure communication between your ERP server and your new NETSTOCK instance

  • A connector is installed on your ERP server that communicates with the Comms server, to send and receive data

Security considerations:

  • Each NETSTOCK instance is created for a specific customer and is completely separate from other customers – there is no way for one customer to access the data of another

  • A secure 2048-bit public/private key combination is generated on the ERP server:

    • The private key never leaves the ERP server

    • The public key is sent to the Comms server via a web service call and is stored in the customer’s unique instance

  • All communication channels are encrypted with the public key and decrypted with the private key, meaning data cannot be intercepted between:

    • The ERP server and the Comms server

    • The Comms server and the App server

  • The comms server is secured by opening only necessary ports and firewalls, keeping your public key safe

  • The following firewall rules have to be set:

    • Port 80 (TCP) open from your ERP server to our Comms server

    • Port 443 (TCP) open from your ERP server to our Comms server

    • Remember, if you’re working through a WAN, then the same rules have to be set between your LAN, and your WAN, and between your WAN and our Comms server

    • You can also make use of a proxy server

Sending data from the ERP server

On a daily basis, data is extracted from your ERP server and sent to NETSTOCK.

On the ERP server, the connector:

  • Runs the extracts against your ERP, creates standard csv files and compresses them using bzip2

  • Communicates with the Comms server using Secure FTP and your private key to send the compressed data via the encrypted channel, followed by an end of transmission web service call

On the Comms server:

  • The data is sent to the App server using Secure FTP

On the App server:

  • Upon receipt of the data, the files are unzipped and MD5 validation totals in a meta-data file are compared with the MD5 values computed for each file sent

  • Once it is established that the correct data has been received, the data is imported into your instance of the App

Sending data to the ERP server

On a more frequent basis, data such as recommended orders or custom reports are sent back to the ERP.

On the App server:

  • When a file is created, the Comms server is notified via a web service call

On the Comms server:

  • The file is retrieved using Secure FTP and compressed using bzip2

  • The list of files to be sent to the ERP is updated

On the ERP server:

  • The Comms server is polled via a web service call for new files

  • If new files are available, the ERP server retrieves the files using Secure FTP with its private key

  • The files are uncompressed and the Comms server notified of successful receipt

Data security

NETSTOCK, the company, is the provider of both the NETSTOCK and Sage Inventory Advisor Apps. References to NETSTOCK in the section below are references to NETSTOCK the company.

NETSTOCK’s customers enjoy the following security:

1. Transit security

All data transferred from the on-premise ERP system to our Comms servers are compressed. This data is then sent via the Secure FTP protocol. This data is encrypted in-transit via session keys and symmetric encryption. The software on the ERP system is authenticated on the Comms server using the customer’s unique public key. The private key is never shared.

The same process then happens to forward the information from the Comms server to the appropriate cloud App server.

For more information about data transmission, see the ERP Connector guide.

2. Data centre security

NETSTOCK only makes use of secure, reputable hosting providers. We only make use of data centres with the following minimum security features:

Security Cameras

Digital security camera system monitors all entries, hallways, and all areas of the lobby and colocation cabinet areas.

Access Control

Entry to the colocation areas requires an access card key.

Air Conditioning

Redundant industrial HVAC units (air conditioners) environmentally control the air temperature and relative humidity in the Colocation Facilities. Cabinets are arranged in alternating hot and cold aisles, with cold air flowing from overhead ducts into the cold aisles, flowing through the cabinets, and exhausting into the warm aisles.

Power, PDUs and Conditioning

Clean, conditioned power is delivered through Power Distribution Units (at least one for each row of cabinets). Each cabinet is individually breakered, so even if one customer has a power issue, other cabinets should not be affected.

Uninterruptible Power Supplies

PDUs are connected to Uninterruptible Power Supplies, which have enough battery power to keep systems running until the generator starts delivering power. All systems undergo regular preventative maintenance.

Power Generators and Fuel

Multiple generators automatically start when outside power is lost, and begin delivering full electric power to the facility within seconds. There should be enough fuel on hand for several days of generator operation at full load, and contracts with local fuel suppliers to promptly replenish when necessary.

Data centres

NETSTOCK makes use of two data centre providers:

Our customers’ data are hosted at the following data centres:

  • North America

    • Linode – Newark, NJ

    • Linode – Fremont, CA

    • Linode – Atlanta, GA

    • Linode – Dallas, TX

  • Africa and Europe

    • Linode – London, UK

    • Hetzner – Nuremberg, DE

    • Hetzner – Falkenstein, DE

  • Australia and New Zealand

    • Linode – Tokyo, JP

    • Hetzner – Nuremberg, DE

    • Hetzner – Falkenstein, DE

3. Storage security

Our servers are all behind firewalls with strict rules in place.

Back-end logins into our servers can only happen with RSA keys, and not via passwords. This means that personnel of NETSTOCK’s access to our back-end servers can be revoked at any time

Our servers are protected from brute-force attacks by automatically banning anyone with 3 failed login attempts for an hour. This happens at the firewall level.

All the OS and application software are patched weekly for any security vulnerabilities.

4. Data isolation

Every customer’s data is completely isolated from every other customer’s data, by using a seperate Database to store their data in.

Similarly, every customer accesses the NETSTOCK service using a unique URL for that customer. A user’s login credentials can never work on another customer’s instance of NETSTOCK.

5. Backups

All data on all servers are backed up every 24 hours. Full backups are retained for 14 days. Any customer’s data can be restored, and depending on the size of the customer’s data the restore will take up to 4 hours to complete.

In case of a catastrophic server failure, new VPSes are spinned up, and customer data restored. The longest a customer will be without a working NETSTOCK system is 48 hours. Typically it’s less than 8 hours.

Backups are stored in a geographic separate data centre, so that a data centre disaster doesn’t affect both the operational servers and the backup servers.

As NETSTOCK is not a mission-critical system, we do not offer automatic fail-over to stand-by servers. This also keeps the monthly cost down for our customers.

Backups are stored and transmitted encrypted.

6. Encryption

All access to a customer’s instance of NETSTOCK goes over the https protocol, using secure TLS versions. Our SSL certificates are signed by trusted CAs. All requests to our web app are protected against Cross-Site Request Forgery.

This means that Man-In-The-Middle attacks are exceedingly difficult to perform. No-one can read our customers’ information whilst in-transit to and from our web servers.

7. Account security

A Password strength checker is used in the App to ensure that weak passwords cannot be selected when creating and resetting passwords.

Passwords are stored hashed and salted using a cryptographically secure algorithm. This means that even if the password hashes are obtained, they cannot be used to log into NETSTOCK.

Accounts are locked out after a defined number of unsuccessful attempts to mitigate brute force attacks. The Customer’s administrator may choose to receive alerts for failed login attempts for the Customer’s user accounts, so that these events can be confirmed to determine whether the login failure was due to legitimate use or malicious attempts to login into the App.

All sessions are automatically logged out after a period of non-use, helping to guard against unauthorised usage of a logged-in system.

Access to support applications that may contain Personal Data or Account data is carefully managed. Our employees are required to manage their credentials using the enterprise password manager that we have provided for this purpose. The password manager not only securely stores credentials, but also generates secure passwords of sufficient complexity and length, and ensures that passwords are not reused across platforms.

8. Security awareness training

Our employees receive regular security awareness training ensuring that they are taught how to work safely online, how to keep their devices safe, how to recognise and avoid information security threats and how to comply with our internal security policies, designed to keep your information safe.

Our employees are also trained to identify security incidents and how to report them in an attempt to reduce the impact and severity thereof.

9. Data retention

In the case that a customer cancels their NETSTOCK subscription, we retain an archive of the customer’s data for three months. This allows for an easier re-instatement of the service, if requested. After three months the data will be deleted forever, even from our backup servers. A full dump of a customer’s data is available upon request in the three month period.

10. Risk management

We follow a risk based approach to security ensuring the ongoing identification, assessment and mitigation of risks to the organisation’s information assets, in order to reduce the probability and impact of their occurence.

11. Incident response

We have developed incident response capability, including formal policies, procedures and training for our employees to ensure that we are able to detect incidents rapidly, minimise loss and destruction, mitigate weaknesses that have been exploited, and restore services in reasonable time frames. The intention is to reduce the probability and impact of incidents that have the potential to occur or have already occurred.

12. Security monitoring and threat prevention

We have implemented security monitoring tools, some of which also have the capability to respond to and stop attacks. These tools include but are not limited to intrusion detection and prevention, behaviour analysis, malware detection, network firewalls and a web application firewall. This ensures early detection of malicious activity and contributes to our response capability.

Full logging is implemented for all systems.

13. Web development security

The security of our code is very important to us. We have subjected our code to an external code audit, as well as internal risk assessment based on OWASP and have also carried out web application vulnerability scanning using numerous tools designed to identify web application threats.

14. Data confidentiality

All NETSTOCK employees sign non disclosure clauses as part of their employment contract, ensuring that they agree to the legal obligation to retain the confidentiality of all customer data. Employees also receive training to educate them regarding data confidentiality requirements and practices.

Supported order formats

See the list below for the currently supported order formats, for uploading orders back into your host system:

  • Acumatica / MYOB Advanced

  • Fishbowl

  • Generic CSV - format used for NetSuite, MYOB EXO and other integrations

  • Microsoft Dynamics GP

  • Microsoft Dynamics NAV

  • Pixi

  • Pronto

  • Retail Express

  • Sage 100

  • Sage 300

  • Sage 500

  • Sage 1000

  • Sage X3

  • Sage Evolution

  • SAP B1

  • Standard CSV - the default CSV order format

  • Standard XML – XML version of the Standard CSV format