DShell = Distributed Shell Distributed Shell = DShell
DShell is a command-line interface, stand-alone component contained with Dimensigon (DM Core) python module.
It translates all user input to RESTful API requests.
DShell makes it easy to work with JSON and interacting with servers regardless of their location or their number.
How to install DM? Pre-req to use DShell.
- Authentication
- Orchestrations & Actions
- Servers
- Configuration
- Log Federation (LogFed)
- Sync
- Software Library
- Help
Before using dshell, you must specify a user and password to connect.
Alternatively, you can save the credentials into a file to improve your automation or embed your automation with dshell commands.
This file can be used to connect remotely if exposed at OS-Level. "Keep it save, delete it when it is not required."
User can be shared by multiple users.
The prompt of dshell specifies the following:
user@<server>:dshell>
Connect
Inside the Program:
dshell> login
OR at command line level:
dshell -login <user_name>
There is no possibility to specify a password as a parameter for security reasons. In case you need to automate, save the credentials with "login --save" dshell command.
Generate a Credential File
Once logged in. Execute:
user@<server>:dshell>
Default location: $HOME/.dshell
Disconnect
From the dshell, you can use one of 2 different commands:
dshell> exit
Using dshell, you can list orchestrations' details, create, run or delete them.
We use a synonym "orch" to refer to Orchestration.
Orchestrations are filled by actions. Actions have versioning for stability and security reasons.
When you need to update an orchestration (same orch. name), a new version will be created.
List orchestrations
dshell> orch list [--version VERSION] [--detail | --schema]
[--like LIKE | --id IDENT | --name NAME]
- orch_name: Name or alias of the orchestration. If the name already exists, a newer version is created.
- version: You can specify a custom version number to retrieve an older version. Latest version is default.
- all: List all the orchestrations. Summarized way.
- detail: List all details about 1 specific orchestration.
Create an orchestration (Interactive-mode)
It is a guide-process that let you create an orchestration step-by-step.
dshell> orch create -name <orch_name>
- Name: Up to 30 characters.
- The symbol "?" means that the orchestration has not been submitted yet.
- If the process fails, a json is created for later re-submission.
- During the creation process, you can add the steps and actions.
- Action are 2 types, DO (Default) and UNDO.
#Example: dshell> orch create -name install_apache dshell:orch:install_apache?> add step dshell:orch:install_apache?:1> add action [action_duid|action_name] dshell:orch:install_apache?:1:action_name> add param -name <> -value <> dshell:orch:install_apache?:1:action_name> cd .. dshell:orch:install_apache?> add step dshell:orch:install_apache?2> add param -name <> -value <> dshell:orch:install_apache?:2> add action [action_duid|action_name] dshell:orch:install_apache?:2:action_name> add param -name <> -value <> dshell:orch:install_apache?:2:action_name> cd .. dshell:orch:install_apache?:3> add undo action [action_duid|action_name] dshell:orch:install_apache?:3:action_name> add param -name <> -value <> dshell:orch:install_apache?:3:action_name> cd .. dshell:orch:install_apache?> preview dshell:orch:install_apache?> submit ......Orchestration successfully created.
Create an orchestration (Batch-mode)
dshell -login <role_name> -orch_create -name <orch_name> -json <path_to_json>
It allows you to script the creation easily.
Load orchestration
In order to "modify"(create a new version) of an orchestration from dshell, you can simply load the orchestration from the catalog to navigate and modify the elements you need (add/del step parameters, add/mod/del actions, etc)
dshell> orch load -name <orch_name> dshell:orch:orch_name>
Run an orchestration (interactive-mode)
dshell> orch load -name <orch_name> dshell:orch:orch_name> run
OR
dshell> orch run -name <orch_name> [-version <version_number>]
- It allows the external automation of the process.
Run an orchestration (batch-mode)
dshell orch run -json -silent
- Scope key,value is mandatory in JSON.
- If a parameter is not specified in JSON file, it will be requested to the user.
- If a parameter is not specified via interactive tty, the process will fail.
- Before running, it outputs a summary of the full orchestration unless -silent parameter is specified.
Delete an orchestration
dshell> orch delete -name <orch_name> [ -version <version_number> | -latest | all ]
- version: It deletes a particular version of the orchestration.
- latest: It is the default mode. The latest version will be deleted, if no other versions are present, the orchestration will be deleted completely.
- all: It will delete all versions stored from the orchestration.
Export orchestration
dshell> orch export -name [ -version <version_number> | -latest | all ] -json <path_to_json>
It will save the orchestration information to a JSON portable file.
Import orchestration
dshell> orch export -name [ -version <version_number> | -latest | all ] -json <path_to_json>
It will import the orchestration information from a JSON portable file.
#Orchestration Template: (Not final structure) { 'orch_name':'_' , 'params':'{}' , steps {1:'action_name,version': custom_params} }
Actions
Action cannot be deleted or modified. You can evolve actions with a newer version but older orchestrations will remain using the older action versions unless you change the orchestration explicitly.
Create an action (Interactive-mode)
DShell will create a new action (newer version if Action Name already exists).
dshell> action create <action_name> -type <shell|ansible|python> -code <path_to_file>
- action_name: Up 30 characters.
- type, Supported types:
- shell: a Shell script in any language (bash,csh,ksh) that it will be support by your target host/s or scope.
- ansible: Playbook. Code must be an ansible playbook.
- python: A Native Python code. Code must be a python file. Modules should be available remotely.
- code: Specify a file locally that contains the code to be performed by the action to a specific scope.
Create/Import an action (Batch-mode)
It requires a credentials file. By default, the the credential file resides in "$HOME/.dshell/cf".
# dshell -login <role_name> -newaction <action_name> -type -code <path_to_file>
Servers are added automatically to the Dimension as they are bootstrapped by indicating the keyring file and 1 reference server. Read more in Concepts and Encryption sections.
If no reference server neither keyring is specified, it means you are deploying a new Dimension.
With dshell, you can list servers and ping their status remotely.
List all servers
dshell> server list -all
List a specific server
dshell> server list -name <server_name>|<server_uuid>
Ping a Server
dshell> server ping -name <server_name>|<server_uuid>
Granules
A granule is an alias for a group of servers. It servers to make operations that affect the selected server group.
Create a Granule
Granule names are case insensitive. The names will be always saved with uppercase letters.
dshell> granule add -name <granule_name> -servers <comma_separated_servers_list_or_serversUUIDs>
Modify a Granule
The modification of the granule by adding or removing more servers, it will not affect the orchestrations already deployed.
For Example, a 3-Servers granule called "servers_svca" for a service "A" to add a fourth server to a granule, it will affect the granule "servers_svca" but it will not affect he service "A" currently deployed in 3 servers instead of 4. If your target is to scale-out your service "A" to 4 servers, you should execute an orchestration against your service "A" indicating the new granule "servers_svca", this will result in a 4 servers after it finishes.
dshell> granule modify -name <granule_name> -servers <comma_separated_servers_list_or_serversUUIDs>
Delete a Granule
dshell> granule delete -name <granule_name>
The deletion of the Granule will NOT affect the current orchestrations or services. The services will continue to work using the resolved array of servers in the deleted granule.
In Dimensigon (DM), you can modify global parameters and local parameters via dshell.
Global parameters affect all servers part of the Dimension.
Local parameters affect the current server you are connected to.
Parameters can be modified only by the administrator.
Parameters List |
|||||
Parameter name | Data Type | Default (Unit) | Global-modifiable | Short Description | Example |
scope | string | Null (Server|Granule|"global") | No | It defines the target group of your subsequent commands | set scope group1 |
logging_repo | string | Null | No | Specifies the Repository server for Log Federation | set logging_repo server123 |
logging_purge | number | 60 (days) | Yes | Number of days to delete inactive logs at Logging Repo | set logging_purge 7 |
entryption | system | on / off | No | Change Data Transport Encryption setting |
List all Global parameters
dshell> param list
List all Local parameters
dshell> param list -local
Modify a Parameter
dshell> set param -name <param_name> -value <value> dshell> set scope = <server_name>|<granule_name>|global dshell:scope> set param -name <param_name> -value <value> #It will set your parameter for the selected scope.
The Log Federation module helps proxying logs to a specific machine for log analysis purposes or to share them with a third-party to increase the support response time.
You can subscribe to log updates locally or remotely.
Only administrator can subscribe to remote logs or directories.
The operator can subscribe to local logs and directories.
If the log does not exist, try directory. If the directory does not exist, the command will fail.
Log Target
Upon reception of the log or directories, logs and directories from multiples source servers are organized as follows:
$DM_HOME/logging/<src_host_name>/original/path/<file_name>
You are able to use the Linux function tail or tail -f to watch the incoming records.
The default logging frequency is 5 minutes. The minimum recommended frequency is 2 minutes. The maximum frequency is 24 hours.
Best Practices:
- Subscribe to meaningful logs/files only.
- If a directory has a high volume of files being generated, avoid subscribing or specify a pattern to reduce the number of subscriptions.
- Patterns may include wildcards. Accepted wildcards are: Asterisk, underline and interrogation symbol.
- "*" Asterisk replaces 1 or more characters.
- "_" Underline or Interrogation symbol "?" replaces only 1 character.
- Patterns may include wildcards. Accepted wildcards are: Asterisk, underline and interrogation symbol.
Subscribe to a log
dshell> subscribe log [<server_name/source>] <server_name/destination> <abs_path_to_log>
Subscribe to a directory
dshell> subscribe dir [<server_name/source>] <server_name/destination> <abs_path_to_dir>
Unsubscribe to a log
dshell> unsubscribe log[<server_name/source>] <server_name/destination> <abs_path_to_log>
Unsubscribe to a directory
dshell> unsubscribe dir[<server_name/source>] <server_name/destination> <abs_path_to_dir> [-pattern <pattern>]
Synchronize configuration files between servers, close or distant via proxying/mesh.
Check syntax on dshell
The Software Library helps making available remote Software locally when required.
You can pull a Software as a step inside your orquestration with a single command.
Software availability is checked every 24 hours.
Add a Software to the Library
dshell> software add <absolute_path> -name <alias_of_the_software> [-family <family_name>] [-server <server_name>|<server_uuid>] dshell> software add -json <abs_path_json>
- Name: Up to 30 characters without spaces.
Try to specify a linux-friendly name or an unique alias that you can easily refer from an orchestration.
- Family: User-defined string.
- Server: If not specified, the software must pre-exist locally from the server you are executing dshell. If specified, Software must be available at destination "server".
- Json: You can add a Software with a command inside or outside dshell by specifying the absolute path to a JSON file.
Pull a Software locally
dshell> software pull -name <alias_of_the_software> -dir <abs_dir_destination_path> dshell> software pull -json <abs_path_json>
- Name: Up to 30 characters without spaces.
Try to specify a linux-friendly name or an unique alias that you can easily refer from an orchestration.
- Dir: the destination directory to place the software to the current server you are logged in or you have set the SCOPE.
- Json: You can pull a Software with a command inside or outside dshell by specifying the absolute path to a JSON file.
Pull a Software to a remote node or granule
dshell> software pull -name <alias_of_the_software> [-server <server_name>|<granule_name>|<server_uuid>] -dir <abs_dir_destination_path> dshell> software pull -json <abs_path_json>
- Name: Up to 30 characters without spaces.
Try to specify a linux-friendly name or an unique alias that you can easily refer from an orchestration.
- Family: A family must pre-exist. This serves to short the software listing.
- Server: If not specified, the software must pre-exist locally from the server you are executing dshell. If specified, Software must be available at destination "server".
- Dir: the destination directory to place the software at remote target/s.
- Json: You can pull a Software with a command inside or outside dshell by specifying the absolute path to a JSON file.
List Software
You can all the Software available in Software Library, Software by Name or by Family.
dshell> software list
dshell> software list -name <alias_of_the_software>
dshell> software list -family <family_name>
Example:
dshell> software list family db
DUID Name Family
--------------------- ------------------------------- --------------
201908081008002138418 tibero6_linux64 db
Filename
----------------------------------------------------------------------
tibero6-bin-FS06_CS_1902-linux64-168227-opt-20190720060807.tar.gz
Size T. Location
----------------------- -- ------------------------------------------
378.0M G db_cluster_1
DUID Name Family --------------------- ------------------------------- -------------- 201904142130488549291 oracle193_linux64 db Filename ---------------------------------------------------------------------- LINUX.X64_193000_db_home.zip Size T. Location ----------------------- -- ------------------------------------------ 2.849G S server1db
Remove a Software from the Library
The removal of the Software does not delete the Software from filesystem at the source server or granule.
dshell> software del -name <alias_of_the_software> dshell> software del -json <abs_path_json>
- Name: Up to 30 characters without spaces.
- Json: You can add a Software with a command inside or outside dshell by specifying the absolute path to a JSON file.
When using dshell, you can get syntax help and a short description of each command by typing the following:
dshell> help
DShell - Distributed Shell (DM) - Dimensigon © 2017-. All rights reserved.
dshell commands:
set
login
keyring
service
software
subscribe
unsubscribe
orch
action
help
Try "help <command>" to get further information.
dshell> help <command>
Example
dshell> help orch list
© Dimensigon. All rights reserved.