MySQL tips and tricks

Restore the DB

If you have sql.gz archive

1 - create target database on server

2 - zcat mydb.sql.gz | mysql -u root -h localhost -p mydb

Find the data location

1 connect to mysql;

2 select @@datadir

Micro Continuous Deployment System

Hi Everyone,

Hope you have a great day, Today we going to build the micro deployment system.

What we need:
- ubuntu server (root access)
- git repository with your code

1. SSH to your favorite Ubuntu server.

ssh -p 22 shmalex@ubutunserver

2. Clone project

git clone https://github.com/shmalex/wifispymap.git

3. Check if your project is working and configure everything you need.

3.1 The main trick in the successful deployment of the system - all configurations should be in a file that does not see LIVE.

This can be done in many ways:

1. place the configuration in the directory above
2. keep all the plates in the same environment
3. keep you configuration in DB.... (but you need to store the connection string to the db...)

3.2 Double check that you projects folder is 'clean' by running command:

git status

3.4 It should show you that there are No/0/ZERO changes

3.4 By knowing that not all projects are perfect, I might need to keep making changes in your project and repeat step 2 and 3 until you achieve the resuls in 3.4.

All good?

ok, now the Continues Deployment part

When you have done all steps in 1,2,3 - the CD is very easy, let's make scheduler that run the some script, that script will contain the commands to pull the project, and other commands that makes our project apply thous changes.

4 - Create 'Deployment' script:

# open home folder
$ cd
# craete file
$ echo '#!/bin/bash' > deployment.sh
# edit the file
$ nano deployment.bash

Ok, now the tricky part.

we need to specify full path to the folders and files in bash script. we can not use the '~/wifispymap' path.

this is my script: 

#!/bin/bash
cd /home/shmalex/wifispymap/
git pull

4.2 make that file executable

chmod +x deployment.sh 

Now let's test it!

1 Test permissions - 

run the command 

$ ls -ahl | grep 'deployment.sh'

It will show the file's permission, we are interested in the x

-rwxrwxr-x  1 shmalex shmalex   50 Jun 15 14:22 deployment.sh

#2 test from home directory

$ cd$ deployment.sh

Already up to date.

#3 test from the root

$ cd /
$ /home/shmalex/deployment.sh

Already up to date.

100% passed.

5 - Convert "Deployment" into ''Continuous Deployment"

We need to use the scheduler,

$ sudo crontab -e

Add new line that will run our script every 5 minutes

*/5 * * * * /home/shmalex/deployment.sh

5.1. Test, following line will show you your changes

$ sudo crontab -l 

5.1 Test

Scheduler adds lines to the syslog file, where we can find our deployment script

$ grep CRON /var/log/syslog

We almost done!!! :)

6. And last test - let's make changes and find them on our server :)

6.1 - make any changes in project, i will update the readme.md file

6.2 - monitor the cron job with command

$ grep CRON /var/log/syslog

...

Jun 15 14:40:01 shmalex-dellpc CRON[14218]: (root) CMD (/home/shmalex/deployment.sh)

Jun 15 14:40:03 shmalex-dellpc CRON[14217]: (CRON) info (No MTA installed, discarding output)

... 

6.3. check the file with text editor, or by running the ls -hls command to check the time of changes file.

Have fun! and feel free to ask any questions :)

Reading materials

1. https://ryanstutorials.net/bash-scripting-tutorial/bash-script.php

2. https://askubuntu.com/questions/650387/how-do-i-create-a-task-scheduler-in-ubuntu

3. https://stackoverflow.com/questions/8779951/how-do-i-run-a-shell-script-without-using-sh-or-bash-commands

How to make Jupyter Notebook full screen

Problem:

The Jupyter Notebook has 900 pixels in width and don't use all the "potential" of your screen.

Especialy when you try to make a presentation of the big screen and you have to Zoom It. (CTRL + MOUSE_SCROLL-UP)

That is exactly where you need the "Full Potential" of you screen.

You play with styles in the Browser's Developer Tool, but this effort is not reusable.

Solution:
Use the Stylebot Extention for Chrome

This extension will save your css and apply it to site when you visit it.

Steps to use
1. Install it
2. Navigate Jupyter Notebook page
3. Open the Stylebot

4. At the bottom of the panel click "Edit CSS"

5. Copy Paste this:
#notebook-container {
    padding: 0px;
}
.prompt {
    min-width: 10ex;
}
div.container {
    width: 95%;
}

so it should look like this:

If you have trouble on "pasting" - just retype this css ;-)


DONE! now it looks way batter - enjoy your presentation!

FYI - the Jupyter Notebook page handles most of the keyboard shortcuts so it's could be tricky to work with Stylebot on JN page.
I recommend to adjust your styles in the Developer Tools and the copy&paste the result into Stylebot.

GL feel free to ask any questions.

WinDBG + VS + DevExpress

.loadby sos clr
.load E:\Diagnostics\sosex.dll
>>  run !bhi
.sympath srv*c:\mss*http://msdl.microsoft.com/download/symbols
.sympath+ E:\WebsiteCode\Live\Platform\Velocity_DigitalHandler\NEWHANDLER\bin

.symfix
.reload /f /i
!sym noisy


!dlk (dead locks)
!dlk
!EEStack -- clr stack

Sampling Distribution

Sampling distribution

Imagine you have some data and you and you would like to know it's properties, like - mean, or average or something else - a Point Estimate. But you don't have a way to work with full data set (it's to large, or it's not fully available to you). But you have access to some portion of it.

To have an good aproximate value of that data you can make the Sampling Distribution.

The sampling distribution represents the distribution of the point estimates based on samples of a fixed size from a certain population.

 It is useful to think of a particular point estimate as being drawn from such a distribution.

To make that point more visible I made the set of tests to show that it really works - the sample size is our variable that experiment.

import mysql.connector
import pandas as pd
import numpy as np
from scipy.stats import bernoulli, binom, poisson, norm, uniform, beta
import matplotlib.pyplot as plt

Distributions and how to plot them with SciPy

Intro

To make the plotting is simple (not that painful), in that post I will make few plots with different distributions.

First importing the libraries. I will use the matplotlib for plotting the data. SciPy one of the core libraries for data science calculations:

import numpy as np # we will use it for generation the arrays
from scipy.stats import bernoulli, binom, poisson, norm, uniform, beta
import matplotlib.pyplot as plt

To make printing comfortable a made the printing function for Mean, Variance, Skew, Kurt:

def print_mvsk(*args):
    t = args[0]
    mean, var, skew, kurt, = float(t[0]),float(t[1]),float(t[2]),float(t[3])
    sd = np.sqrt(var)
    print(f'mean:{mean:.4f}\tvar:{var:.4f}\tskew:{skew:.4f}\nsd:{sd:.4f}\tkurt:{kurt:.4f}')

Performing Accurate Decimal Operations

In python the decimal calculations using the IEEE 754 algorithm.
>>> a = 4.1
>>> b = 2.2
>>> a + b
6.300000000000001

that means that
>>> (a + b) == 6.3 
False
Because python's float type stores data using the native representations.

If you want more accuracy Decimal class from decimal module can help you with that.

>>> from decimal import Decimal
>>> a = Decimal('4.1')
>>> b = Decimal('2.2')
>>> a + b
Decimal('6.3')
>>> (a + b) == Decimal('6.3')
True