Wednesday, December 12, 2012

search file - Improve performance

First in Visual Basic .NET


When you program, if the task in your loop is the same over and over, you might want to recycle some code from your program. This will multiply the power of your power by 2, by 5 or 10 and make your code shorter.
Here is an example from a previous post. The sample code in Visual Basic will search for all the files with a pattern in a directory/subdirectory and list them in a ListBox.  Isn’t that great? Yes and no. Even with the best tactic to speed up your code, you will wait forever if the directory has a lot of subdirectories with a lot of files.

Old code: search file without recursive feature


    Private Sub findfiles()

        Dim dirPath As String
        Dim path_valid As Boolean
        path_valid = True


        dirPath = "c:\temp\" ' temp = no access problem
        dirPath = "c:\System Recovery" ' System Recovery = access problem




        '----------------------------------------------
        ' CHECK PATH IF VALID
        '----------------------------------------------
        'trim the sPath to remove space before and after the String
        dirPath = Trim(dirPath)

        'check if sPath is not zero length
        If Len(dirPath) > 0 Then

        Else
            path_valid = False
        End If

        'check for invalid char
        ' \ * | : ? < > / except the \ because is a directory seperator
        ' and * that means to ignore whatever inside the other chars. (optional)
        If InStr(dirPath, "|", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, ">", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, "<", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, "?", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, ":", CompareMethod.Binary) = 2 Then
            'very basic way to check if there is a ":" in the Path.
            Dim str_temp As String = dirPath.Substring(2, Len(dirPath) - 2)
            If InStr(str_temp, ":", CompareMethod.Binary) > 0 Then
                path_valid = False
            End If

        End If
        If InStr(dirPath, "*", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        'be careful with non-english caracters


        '----------------------------------------------
        ' SECURITY
        '----------------------------------------------
        Dim oDirectorySecurity As System.Security.AccessControl.DirectorySecurity
        Dim oAuthorizationRuleCollection As System.Security.AccessControl.AuthorizationRuleCollection
        Dim oRule As System.Security.AccessControl.FileSystemAccessRule
        Dim Security_valid As Boolean

        Security_valid = False
        oDirectorySecurity = IO.Directory.GetAccessControl(dirPath)
        oAuthorizationRuleCollection = oDirectorySecurity.GetAccessRules(True, True, Type.GetType("System.Security.Principal.NTAccount"))

        For Each oRule In oAuthorizationRuleCollection

            If oRule.FileSystemRights = Security.AccessControl.FileSystemRights.FullControl Then
                If InStr(oRule.IdentityReference.Value, My.User.Name) > 0 Then
                    Security_valid = True
                End If
            End If

        Next

        '----------------------------------------------
        ' SECURITY
        '----------------------------------------------
        Dim Is_Folder_shortcut As Boolean
        Is_Folder_shortcut = False
        If dirPath.StartsWith("c:\Users", True, System.Globalization.CultureInfo.CurrentCulture) Then
            'very basic way to force you to be very carefull with that folder
            Is_Folder_shortcut = True
        End If

        '----------------------------------------------
        ' IF ALL OK, THEN DISPLAY THE RESULT
        '----------------------------------------------
        If oForm2 Is Nothing OrElse oForm2.IsDisposed Then
            oForm2 = New Form2
            oForm2.Show()
        End If
        oForm2.ListBox1.Items.Clear()
        If path_valid = True And Security_valid = True And Is_Folder_shortcut = False Then

            For Each foundFile As String In My.Computer.FileSystem.GetFiles( _
                    dirPath, FileIO.SearchOption.SearchAllSubDirectories, "*.dll")

                oForm2.ListBox1.Items.Add(foundFile)

            Next
        Else
            MsgBox("path_valid =" & path_valid & vbCrLf & "Security_valid =" & Security_valid)
        End If

    End Sub



Now, this is a new code with minor change. All modifications are highlighted in yellow.


    Private Sub findfiles(Optional dirPath As String = "c:\temp\", Optional String_to_Search As String = "*.dll")

        'Dim dirPath As String ' you need to put in argument a "top" directory (optional if you want)
        Dim path_valid As Boolean
        path_valid = True

        'dirPath = "c:\temp\" ' temp = no access problem
        'dirPath = "c:\System Recovery" ' System Recovery = access problem




        '----------------------------------------------
        ' CHECK PATH IF VALID
        '----------------------------------------------
        'trim the sPath to remove space before and after the String
        dirPath = Trim(dirPath)

        'check if sPath is not zero length
        If Len(dirPath) > 0 Then

        Else
            path_valid = False
        End If

        'check for invalid char
        ' \ * | : ? < > / except the \ because is a directory seperator
        ' and * that means to ignore whatever inside the other chars. (optional)
        If InStr(dirPath, "|", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, ">", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, "<", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, "?", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        If InStr(dirPath, ":", CompareMethod.Binary) = 2 Then
            'very basic way to check if there is a ":" in the Path.
            Dim str_temp As String = dirPath.Substring(2, Len(dirPath) - 2)
            If InStr(str_temp, ":", CompareMethod.Binary) > 0 Then
                path_valid = False
            End If

        End If
        If InStr(dirPath, "*", CompareMethod.Binary) > 0 Then
            path_valid = False
        End If
        'be careful with non-english caracters


        '----------------------------------------------
        ' SECURITY
        '----------------------------------------------
        Dim oDirectorySecurity As System.Security.AccessControl.DirectorySecurity
        Dim oAuthorizationRuleCollection As System.Security.AccessControl.AuthorizationRuleCollection
        Dim oRule As System.Security.AccessControl.FileSystemAccessRule
        Dim Security_valid As Boolean

        Security_valid = False
        oDirectorySecurity = IO.Directory.GetAccessControl(dirPath)
        oAuthorizationRuleCollection = oDirectorySecurity.GetAccessRules(True, True, Type.GetType("System.Security.Principal.NTAccount"))

        For Each oRule In oAuthorizationRuleCollection

            If oRule.FileSystemRights = Security.AccessControl.FileSystemRights.FullControl Then
                If InStr(oRule.IdentityReference.Value, My.User.Name) > 0 Then
                    Security_valid = True
                End If
            End If

        Next

        '----------------------------------------------
        ' SECURITY
        '----------------------------------------------
        Dim Is_Folder_shortcut As Boolean
        Is_Folder_shortcut = False
        If dirPath.StartsWith("c:\Users", True, System.Globalization.CultureInfo.CurrentCulture) Then
            'very basic way to force you to be very carefull with that folder
            Is_Folder_shortcut = True
        End If

        '----------------------------------------------
        ' IF ALL OK, THEN DISPLAY THE RESULT
        '----------------------------------------------
        If oForm2 Is Nothing OrElse oForm2.IsDisposed Then
            oForm2 = New Form2
            oForm2.Show()
        End If
        'oForm2.ListBox1.Items.Clear() ' recursive search file : don't clear the listBox
        If path_valid = True And Security_valid = True And Is_Folder_shortcut = False Then

            For Each foundFile As String In My.Computer.FileSystem.GetFiles( _
                    dirPath, FileIO.SearchOption.SearchTopLevelOnly, String_to_Search) 'recursive search file : you need to limit your search to 1 single directory : and put extention in search argument

                oForm2.ListBox1.Items.Add(foundFile)

            Next

            For Each foundDirectory As String In My.Computer.FileSystem.GetDirectories(dirPath, FileIO.SearchOption.SearchTopLevelOnly)
                findfiles(foundDirectory) ''recursive search file : re-use "himself"
            Next
        Else
            MsgBox("path_valid =" & path_valid & vbCrLf & "Security_valid =" & Security_valid)
        End If

    End Sub




Here is the explanation and I will try to keep it short. Instead of asking the My.Computer.FileSystem.GetFiles to seach for all files in all subDirectories in 1 single operation, we ask the findfiles function to limit his search on the top level directory.

For all other directory or subdirectory, we re-call the same findfiles function to do the same operation. The program will loop himself until there are no more directory to look for.
Is like asking the computer to play chess with himself.

Because you are able to separate 1 single big search to multiples tiny search, you will be able to multi-thread the while operation.

Good to read:
Previous Post : Search a file
My other related post: Create the Search TextBox
My other related post: Create your Windows 8 Start Menu

Visit my Official Web Site : Check Technologies

About 

I invite you to visit my blog for more articles and leave a comment. Check Technologies represents more than 10 years .... Computer and computer aided design.

Check Technologies ltée
Official Website Check Technologies




No comments:

Post a Comment