1 (edited by jrga 2023-12-12 23:33:17)

Topic: MVD with Business Intelligence (B.I.) tool

I coupled an MVD 5.6 application with a BI tool, which is Qlikview 11 Personal Edition (freeware - link below), according to the video in the link below. Video capture was slow because everything was played on a MicroSD card.


link vídeo = https://youtu.be/PJEo3ch3Dic


I obtained the script below here on the forum, which exports data from the MVD application to a file in CSV format.


I created a dashboard in Qlikview from the CSV file. Below is the script I used in the dashboard, which loads data from the CSV file, which is used to create tables, graphs, etc. in Qlikview,


Qliview's capabilities and possibilities are far above this small example. There are countless tutorials, including videos, on the internet. In addition to the help file (CHM) and the examples that accompany it.


Limitations of the Personal Edition version (taken from the internet): "QlikView Personal Edition is a free version of QlikView for personal use. It is the full QlikView Desktop product and uses the same installation package1. However, there are some limitations and aspects to be observed:


     QlikView Documents: In QlikView Personal Edition, every document is saved with a user key that associates the file with the computer on which it was created1. This means that you cannot use your QlikView documents on other computers, share your QlikView documents with another unregistered user, and cannot open another user's QlikView document1. However, documents created in QlikView Personal Edition can be used with a licensed copy of QlikView Desktop and can also be published to a QlikView Server1.


     File recovery: If you change computers, you will not be able to open documents previously created with QlikView Personal Edition1. In this case, you can choose to recover your files1. QlikView will accept 4 user key changes in a document1. After this, the error message “Key length has reached maximum” will be displayed and the document will no longer open1.


     Support: QlikView Personal Edition users are not eligible for phone or email support from Qlik, but can get excellent support for QlikView from our Qlik1 Community."


Qlikview download link. I downloaded it from the manufacturer's website at the time:
    
https://drive.google.com/file/d/1jeWUs4 … sp=sharing


SCRIPT USED IN THE MVD APPLICATION

procedure frmOverview_bExpCSV_OnClick (Sender: TObject; var Cancel: boolean);
var
   filename: string;
begin
    frm_aviso.Show;
    Form1_bExport; //aciona rotina exportação de lancamento_all_view para arquivo CSV, que será utilizado no qlikview
    filename := 'dashboard_orcamento_jrga.qvw';
    OpenFile(filename);
    frm_aviso.Close;
end;

procedure Form1_bExport;
var Sl : TstringList;
  folder: string;
  TableNames: TDataSet;
  TableCols: TDataSet;
  TableName,ColName,Cols,SelectStmt,ColType: string;
begin
   ProgressBar := TProgressBar.Create(frmMyNotas);
   ProgressBar.Parent := frmMyNotas.pnProgressBar;
   ProgressBar.Width := frmMyNotas.pnProgressBar.Width;
   ProgressBar.Height := frmMyNotas.pnProgressBar.Height;
   ProgressBar.Position := 0;
   ProgressBar.Max := 10;
   Sl := TstringList.Create;
   //SelectDirectory('Select Folder to Export File', './', folder, True);  //Dialog to select folder for which the exported file will be created
   //If Folder = '' then Exit;             // If no folder is selected then cancel the export
   frmMyNotas.pnProgressBar.Visible := True;
   frmMyNotas.lblImpExp.Caption := 'Exporting Database';
   application.processmessages;
   SqlQuery('SELECT name FROM sqlite_master WHERE type="view" and name = "lancamento_all_view"',TableNames); // Retrieve all tablenames from database and load into a dataset
   while not TableNames.Eof do   // Run through each tablename dataset
      begin
        Cols := '';
        SelectStmt := '';
        TableName := TableNames.FieldByName('name').AsString;
        SqlQuery('pragma table_info(' + TableName + ')',Tablecols); // Retrieve column (fields) names for each table
        While not TableCols.Eof do   // Go through each column (field) and concatenate together with comma separation
          begin
             ColName := TableCols.FieldByName('name').AsString;
             ColType := Tablecols.FieldByName('type').AsString;
             If ColType <> 'BLOB' then Cols := Cols + ColName + ','; // Do not select images for exporting
             TableCols.Next;
          end;
        Delete(Cols, Length(Cols), 1);   // Remove ending comma from column list
        ProgressBar.Position := ProgressBar.Position + 10;
        ExportToSl(TableName,Cols,Sl);  // Run Export procedure - pass the tablename and the list of columns to the procedure
        TableNames.Next;  // Go to next table name in dataset
      end;
   Sl.SavetoFile(path_atual+'ExportedData.csv'); // Save the stringlist to a csv file into the specified folder
   ProgressBar.Free;
   frmMyNotas.pnProgressBar.Visible := False;
   frmMyNotas.lblImpExp.Caption := '';
   //ShowMessage('Dados foram exportados para CSV');
   Sl.Free;
   TableNames.Free;
   TableCols.Free;
end;

SCRIPT USED IN QLIKVIEW. Loads data for use with Qlikview graphical objects. The first 10 lines are created automatically. Once the table with the data has been chosen, Qlikview will generate the "LOAD ...FROM" structure

SET ThousandSep='.';
SET DecimalSep=',';
SET MoneyThousandSep='.';
SET MoneyDecimalSep=',';
SET MoneyFormat='R$ #.##0,00;-R$ #.##0,00';
SET TimeFormat='hh:mm:ss';
SET DateFormat='DD/MM/YYYY';
SET TimestampFormat='DD/MM/YYYY hh:mm:ss[.fff]';
SET MonthNames='jan;fev;mar;abr;mai;jun;jul;ago;set;out;nov;dez';
SET DayNames='seg;ter;qua;qui;sex;sáb;dom';

Directory;
Lancamentos:
LOAD TableName, 
     data as "Data",
     tipo_conta as "Iipo Conta", 
     conta as "Conta", 
     categoria as "Categoria", 
     subcategoria as "Subcategoria", 
     categoria &' : '& subcategoria as "Categ.e subcat.",
     cheque as "Nºcheque", 
     grupo as "Grupo fornecedor", 
     fornecedor as "Fornecedor", 
     if(id_tipo_operacao='1','Entrada','Saída') as "Tipo operação", 
     Replace(entrada,'.',',') as "Vlr entrada",
     Replace(saida,'.',',')*-1 as "Vlr saida",
     (Replace(entrada,'.',',') + Replace(saida,'.',',')*-1) as "Valor", 
     Replace(meta_anual,'.',',') as "Meta anual",
     notas as "Notas",
     id_lancamento as "ID lancamento", 
     id_tipo_conta as "ID tipo conta", 
     id_conta as "ID conta", 
     id_categoria as "ID categoria", 
     id_tipo_operacao,
     id_subcategoria as "ID subcategoria", 
     id_grupo as "ID grupo", 
     id_fornecedor as "ID fornecedor",
     checado as "Conciliado" 
FROM
ExportedData.csv 
(txt, utf8, embedded labels, delimiter is ',', msq) 
WHERE len(categoria) <>0;


LOAD data as "Data"
    , year([data]) as Ano
    , month([data]) as Mes
    , day([data]) as Dia
FROM
ExportedData.csv
(txt, utf8, embedded labels, delimiter is ',', msq);

P.S.: in the document settings, in Qlikview, it is possible to create a trigger, which updates the dashboard data, as soon as the QVW file is opened.

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

That looks very cool, if only we could design this kind of stuff within My visual Database!!

3 (edited by jrga 2023-12-17 00:42:48)

Re: MVD with Business Intelligence (B.I.) tool

tcoton wrote:

That looks very cool, if only we could design this kind of stuff within My visual Database!!


You can use Qlikview in any MVD application! I decided to share my experience for two reasons: 1 - to contribute to the forum from where I gained all my knowledge about MVD, from where I downloaded countless projects, which further enriched my learning. 2 - encourage users to continue using MVD, in view of that he has been discontinued.

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

Yeah but the price of the product for commercial use is just prohibitive!!! I have thought of integrating some python script within MVD to do exactly the same thing but for free big_smile I am still working on it...

Re: MVD with Business Intelligence (B.I.) tool

anybody can be share sample MVD application with a BI tool ??

6 (edited by jrga 2024-01-08 15:25:16)

Re: MVD with Business Intelligence (B.I.) tool

hilal wrote:

anybody can be share sample MVD application with a BI tool ??

5 years ago, I created a solution to analyze data, which is part of the attached project and the one in the link https://youtu.be/gUu7wrpQZKY


Not a B.I. and it is light years away from that, but it allows, through a dynamic query, which makes reference to all the project's interrelated tables, to add values, quantities, check the percentage variation, filter by periods, etc.


Contains a simple graph. I called it "Data Crossing". The objective is the same as those that BI seeks: to assist in decision making.


In this project, which I did for my brother, the data was imported from Access and there were blank related fields and that's why I added the option to filter blank fields, but this is unnecessary when the project is built from scratch. Replaces names, addresses and telephone numbers.


Like so many things in MVD, it can be improved.


link for download:

https://drive.google.com/file/d/1kqnN06 … sp=sharing

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

@jrga
I see in the video that you are running the compiled project with wine on Mac OS, did you develop this project via wine too or did you use a PC/Virtual machine? I have a Mac too and I use virtual machines for Windows stuffs.

Re: MVD with Business Intelligence (B.I.) tool

Hi Roberto,


good job.


"frmOverview.TableGrid1.beginUpdate" was lost in the atualiza_overview procedure.
Adding the lost one will speed up the output of data to the table.

9 (edited by jrga 2024-01-05 00:46:25)

Re: MVD with Business Intelligence (B.I.) tool

tcoton wrote:

@jrga
I see in the video that you are running the compiled project with wine on Mac OS, did you develop this project via wine too or did you use a PC/Virtual machine? I have a Mac too and I use virtual machines for Windows stuffs.

Hello tcoton! I used wine itself via playonMac to develop MVD applications on MAC and also on Linux (playonLinux). Please see the link https://myvisualdatabase.com/forum/view … hp?id=8566

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

sparrow wrote:

Hi Roberto,


good job.


"frmOverview.TableGrid1.beginUpdate" was lost in the atualiza_overview procedure.
Adding the lost one will speed up the output of data to the table.

Hi Sparrow, I'll test it. Thanks.

Roberto Alencar

11 (edited by jrga 2024-01-05 18:50:29)

Re: MVD with Business Intelligence (B.I.) tool

jrga wrote:
sparrow wrote:

Hi Roberto,


good job.


"frmOverview.TableGrid1.beginUpdate" was lost in the atualiza_overview procedure.
Adding the lost one will speed up the output of data to the table.

Hi Sparrow, I'll test it. Thanks.


Hello Sparrow! It worked. That is, it accelerated the consultation. Thanks.


procedure atualiza_overview;
//**** aciona geração da consulta e do gráfico de barras
var
    txt_sel, txt_sel1, txt_where: string;
    c,i: integer;
    total_grid: real = 0;
    Results: TDataSet;
    x_value: string;
    cf_y: string;
    y: integer;
begin
    frmOverview.TableGrid1.beginUpdate; //sparrow
....................
...................
...................
    frmOverview.TableGrid1.EndUpdate; //sparrow
end;

P.S.: Looking at the script I found frmOverview.TableGrid1.beginUpdate "lost" and now I understand your message.

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

Sorry Roberto)

that's exactly what I meant). "Lost"


One more thing. Table "bar_data".
This table is used once to form a chart and is erased to form a new chart.
At the same time, the size of the database is constantly increasing.
Pay attention to this feature of SQLITE as a temporary table.

    SQLExecute('DROP TABLE IF EXISTS bar_data_temp');
    SQLExecute('CREATE TEMP TABLE bar_data_temp (eixo_x TEXT, eixo_y REAL)');
    SQLExecute('INSERT INTO bar_data_temp (eixo_x, eixo_y) ...  '+

Technically, SQLite stores temporary tables in a separate temp database.
It keeps that database in a separate file on disk, visible only to the current database connection.
The temporary database is deleted automatically as soon as the connection is closed.


Such a table does not need to be registered in MVD.
I offer it just for information. Perhaps it will be useful in future projects, and perhaps in this one too.

13 (edited by jrga 2024-01-06 17:00:40)

Re: MVD with Business Intelligence (B.I.) tool

sparrow wrote:

Sorry Roberto)

that's exactly what I meant). "Lost"


One more thing. Table "bar_data".
This table is used once to form a chart and is erased to form a new chart.
At the same time, the size of the database is constantly increasing.
Pay attention to this feature of SQLITE as a temporary table.

    SQLExecute('DROP TABLE IF EXISTS bar_data_temp');
    SQLExecute('CREATE TEMP TABLE bar_data_temp (eixo_x TEXT, eixo_y REAL)');
    SQLExecute('INSERT INTO bar_data_temp (eixo_x, eixo_y) ...  '+

Technically, SQLite stores temporary tables in a separate temp database.
It keeps that database in a separate file on disk, visible only to the current database connection.
The temporary database is deleted automatically as soon as the connection is closed.


Such a table does not need to be registered in MVD.
I offer it just for information. Perhaps it will be useful in future projects, and perhaps in this one too.

Hi Sparrow, I converted the graph table to temporary, as you suggested and re-uploaded the file, whose download link is in the original post. Thanks for the suggestions.

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

Hi Roberto.
It needs to be a little different...
It is necessary that the deletion of the table, its creation and filling be done in the "atualiza_overview" procedure. I'll attach the script.

Post's attachments

Attachment icon script.pas 21.38 kb, 28 downloads since 2024-01-07 

Re: MVD with Business Intelligence (B.I.) tool

sparrow wrote:

Hi Roberto.
It needs to be a little different...
It is necessary that the deletion of the table, its creation and filling be done in the "atualiza_overview" procedure. I'll attach the script.


Hi Sparrow, I used the script.pas you sent in the project, uploaded the file and updated the download link in the original post. Thanks again for helping to improve this project. Everyone wins!

Roberto Alencar

Re: MVD with Business Intelligence (B.I.) tool

jrga wrote:
hilal wrote:

anybody can be share sample MVD application with a BI tool ??

5 years ago, I created a solution to analyze data, which is part of the attached project and the one in the link https://youtu.be/gUu7wrpQZKY


Not a B.I. and it is light years away from that, but it allows, through a dynamic query, which makes reference to all the project's interrelated tables, to add values, quantities, check the percentage variation, filter by periods, etc.


Contains a simple graph. I called it "Data Crossing". The objective is the same as those that BI seeks: to assist in decision making.


In this project, which I did for my brother, the data was imported from Access and there were blank related fields and that's why I added the option to filter blank fields, but this is unnecessary when the project is built from scratch. Replaces names, addresses and telephone numbers.


Like so many things in MVD, it can be improved.


link for download:

https://drive.google.com/file/d/1kqnN06 … sp=sharing







Ok ..  Good Job....... Thanks you for sharing ...  I want to try make like that on my project ....